1 / 43

by Software Performance Assurance on Aug 28, 2010

SaaS Testing Overview - Foundation is an introduction to SaaS and SaaS Testing. In subsequent presenation, I will cover SaaS Testing Overview - Expert

spassurance
Download Presentation

by Software Performance Assurance on Aug 28, 2010

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SaaSTestingOverview RamGarg CEO SoftwarePerformanceAssurance(SPA) SoftwarePerformanceAssuranceConfidential

  2. Outline SaaS Introduction Whatis it? WhySaaS? SaaS Challenges KeyChallenges SaaSTesting HowtoapproachSaaStesting? HowSaaSTestingisdifferent? SaaSTest Tools SoftwarePerformanceAssuranceConfidential

  3. WhatisSaaS? SaaSIntroduction SoftwarePerformanceAssuranceConfidential

  4. SaaS:Whatisit? Software-as-a-Service(SaaS)istheapplicationlayerofthecloudcomputingmodel. Application SoftwareasaService(SaaS) Platform PlatformasaService(PaaS) Infrastructure InfrastructureasaService(IaaS) SoftwarePerformanceAssuranceConfidential

  5. SaaSDefinition “Softwareasaservice(SaaS),sometimesreferredtoas"softwareondemand,"issoftwarethatis100%deployedoverthe internet….WithSaaS,aproviderlicensesanapplicationtocustomersasaservice ondemand,throughasubscriptionora "pay-as-you-go"model.” –Wikipediadefinition SoftwarePerformanceAssuranceConfidential

  6. Architecture:ThreeKeyDifferentiators Multi-tenant Efficient Scalable Configurable Key Architecture Differentiators SoftwarePerformanceAssuranceConfidential

  7. WhySaaS? SaaSIntroduction SoftwarePerformanceAssuranceConfidential

  8. TheSoftwareBusinessModelChanges… non-SaaS SaaS Customer Provider SoftwareOwnership Infrastructure managementResponsibility Customer Provider Client-Server Central Installation Larger,lessfrequent Shorter,frequent Upgrades Multi-version Single-version Platform SoftwarePerformanceAssuranceConfidential

  9. ChangingtheBusinessModelcont… non-SaaS SaaS ConfigurabilityRequirement Low High,Critical No Yes Multi-tenancy No Yes Backwardcompatibility Customer Provider Maintenance SoftwarePerformanceAssuranceConfidential

  10. ChangingtheBusinessModelcont… non-SaaS SaaS High,CapEx Low,OpEx Cost,Allocation Closethedeal, LicenseRevenue Providevalue, LackofChurn Salesfocus/Success Initialsale+maintenance OngoingSubscription Profits SoftwarePerformanceAssuranceConfidential

  11. Ecosystem SaaS(Software) PaaS(Platform) IaaS (Infrastructure) Startingpointtoeveryservicewithin cloudcomputingecosystem Higheston thematuritylevel and growthcurveamongstXaaS Highestrevenuegeneratingservice withincloudmodelfornextseveral years

  12. SaaSCatalysts Intrigueof accessto unlimited Computingandstorage resources,madeavailableandpaidfor onlyas needed(Payasyougo) Decreasingcostof bandwidth Customerfrustrationwithtraditionalsoftwarebuyingcycle (license,maintenancecontract,upgrades) Perceived increaseincontroloverthevendorrelationship EarlySaaSsuccessstories– Salesforce.com,WebEx,etc. Demonstratebroadervalueproposition Youfocuson yourapplicationandnot oninfrastructure SoftwarePerformanceAssuranceConfidential

  13. SaaSEarlyAdoption SaaSIntroduction SoftwarePerformanceAssuranceConfidential

  14. SaaSAdoption:Verticals AMI-Partners„report- "Software-as-ServiceHotSpots”:Education, Healthcare,andFinancial" Adoptionrate of SaaSinSMBis higher thanlargeenterprises All researchpoints outthat SaaShas beenadopted heavilyin: HRapplications(ERP,Salesforceautomation, eRecruiting) OnlineCollaboration(Web conferencing) CRM Healthcare In-Statresearchon verticalmarketadoption showsthehealthcare marketiscurrentlymost satisfiedwithhosted applications,witheducationidentifiedasan importantpotentialtarget marketalso. SoftwarePerformanceAssuranceConfidential

  15. SaaSAdoptionFactors (source:Forrester) Availability&Reliability(71%) FastDeployment(45%) Reduceddependencieson internalIT(24%) Flexibleconsumption(35%) Simplifiedpricingmodel(32%) SoftwarePerformanceAssuranceConfidential

  16. The SaaS Market is andwill Continue to Grow… SoftwarePerformanceAssuranceConfidential

  17. Todayandtomorrow… SoftwarePerformanceAssuranceConfidential

  18. Todayandtomorrow… SoftwarePerformanceAssuranceConfidential

  19. Todayandtomorrow… SoftwarePerformanceAssuranceConfidential

  20. Challenges SaaSChallenges SoftwarePerformanceAssuranceConfidential

  21. PrimaryConcernswiththeSaaSModel Securityconcerns Concernsovercompliancewithstandards Concernsoverintegrationwith legacyapplications Concernsoverreliabilityandrobustnessofservice Notcomfortablewith puttingmission-criticalapplicationonshared infrastructureandhave accessover internet SaaScannotbe configuredtomeetmycorporation‟sall specific needs SoftwarePerformanceAssuranceConfidential

  22. NumerousChallenges Security Performance Availability SLA Compliance with Standards DataIntegrity Privacy Regulatory Lackof confidence Integration Multi-tenancy SoftwarePerformanceAssuranceConfidential

  23. SaaSTesting SoftwarePerformanceAssuranceConfidential

  24. WhySaaS Testing? SaaSTesting SoftwarePerformanceAssuranceConfidential

  25. WhySaaS Testing? TheaverageenterpriseIT departmentdevotesupto50 percentofitsentiretechnology infrastructuretodevelopment andtest,withupto90percent oftheavailabletest infrastructureremainingidle. IBMResearch Adelayinwebsitepage loadingbyeven onesecond canimpactconversionsby 7%andcustomersatisfaction by16%. AberdeenGroup Cloud Testing Thoseusingcloudfacilitiesto runapplications,49.8%said theyweredoingso experimentallyorfor prototyping;28.6%fornon- criticalbusinessapplications and21.6%forbusiness criticalapplications EvansDataSurvey Theaverageonline businesshad8daysof downtimeayeardueto performanceissues leadingtolostrevenues AberdeenGroup SoftwarePerformanceAssuranceConfidential

  26. Trendleadstotrend… OverallgrowthinCloudComputing(investment, revenue,ROI,marketcap,etc.)isleadingtoa markedincreasein adoptionofSaaSTesting

  27. WhatisSaaS Testing? SaaSTesting SoftwarePerformanceAssuranceConfidential

  28. ClassifyingSaaS Testing “TestingTheCSlaoauSd”” Testingapplicationswhicharein cloudor migratingtocloud. Testing theSaaS Testtheperformance,reliability& securityofapplications Cloud/ SaaSTesting “Te“sTtiensgtiSngaaUSsiUngsinCgloCudlo”ud”Testingapplicationsusingthe cloud-basedhardware,software, infrastructureandcomputing resources. Testing SaaS Using Cloud Testsperformedlikeperformance, load&stress,securityand compatibility SoftwarePerformanceAssuranceConfidential

  29. HowSaaS TestingisDifferent Easier Challenges • Noclient-server • installation • Nomultiplatformsupport • Nomultipleversion • upgrades • Nobackwardcompatibility • Frequentliveupgrades • Toomanypages • Multipleprivileges • ImmatureSaaS-specific • testingtools • Accesspartitionscritical SoftwarePerformanceAssuranceConfidential

  30. Approach SaaSTesting SoftwarePerformanceAssuranceConfidential

  31. SaaSTestingPreparation Prepare TestPlan andTest Suite Assess Testing Needs Understand Usage Patterns PrepareTest Environment Prepare TestData SoftwarePerformanceAssuranceConfidential

  32. SaaSTestingAreas InfrastructureTesting Network testing Application testing Secured SaaS Application EnsureSaaStestingdoesnotdamage thesharedinfrastructure SoftwarePerformanceAssuranceConfidential

  33. SaaSFunctionalTesting • PerformManualandExploratorytest(similartotraditional applications) • CheckBrowsercompatibility • Performregression • SaaSregressionmustcoversignificantmorefunctionalityas single defectwillimpactall thecustomers • Functionaland regressiontestautomation. • Thisis importantto comprehensivetesting cost-effectivelyas above pointindicatesthat,incaseof SaaS,regressionshouldcover more functionality. • TestSaaSon cloud-basedstagingenvironment • Testingonin-houseenvironmentand thendirectlydeployingoncloud- basedproductionenvironmentmayresult in chaos • Testin amulti-userenvironmentwheredifferentusersare performingdifferent activities. SoftwarePerformanceAssuranceConfidential

  34. SaaSSecurityTesting • Testapplicationsecurity • CrossSitescripting, SQLInjection, HTTPheaderinjection • Testnetwork security • Secured communication, Backupandstoragepolicies,Controlled accessto sites andservers • Testfor possibleattacks • Testroles andprivilegesinmulti-tenant environment • Manyrolesandprivilegelevels • Testall the upgrades to detect broken security, • Ensuresecurity patchesare not missed • Testdata security / integrityamong multipletenants SoftwarePerformanceAssuranceConfidential

  35. SaaSSecurityTesting cont … • Identifyscenarioswhere applicationmight be vulnerable • TestPCIcompliance • Testcookiesetting, SSLconfiguration, information leak, etc. • Logerrors, warningsandsuspiciousrequests • Teststampering withthelicensingmechanism; gainingunauthorizedaccess to applicationdata, editing registry entitiesor configurationfiles SoftwarePerformanceAssuranceConfidential

  36. SaaSPerformanceTesting • Testperformanceextensivelyfirst time andfor eachupgrade • SaaSis revenuegeneratingandbusinessimpactingapplications. • Single performanceissuewillimpactallyourcustomers • Testperformanceextensivelyas SaaSneedto handlelargenumberofusers andwedon‟thaveluxuryof re-bootingor goingdownevery once ina while • Testloadcharacteristics,rampup rate,protocols,load origination, • Establishabilityto testperformanceof yourSaaSisolated fashionsothatnetworkrelatedissuescanbe identifiedand passedtonetworkinfrastructureandinternet providers • TestkeymetricsandSLAs • Testperformancefor mobileusers SoftwarePerformanceAssuranceConfidential

  37. In-ServiceUpgradeTesting • Abilitytoupgrade(fullrelease,patches)systemwithzero- to-minimumdowntimeisessential • Globalusersmakethiscomplicatedassomewhereinthe • worldsomeonemightbeaccessingyourapplication • Thoroughlytestinstagingenvironmentbeforeperform upgradeassthisisoneofthemostcomplexfeatures • RemoteAccessibilityand Usage • Global access with acceptableresponsetime • 24hoursavailability SoftwarePerformanceAssuranceConfidential

  38. DisasterRecovery • Shutdowntobringup applicationquickly: serverconfiguration; installand deploysoftware;restoredata • Rollbacktolastknownstabledata • Localization • Makesureit supportsinternationalcharactersasSaaS applicationcanbe accessedfromanywherein the world • Optimizebysupportingtargetmarketlanguage SoftwarePerformanceAssuranceConfidential

  39. TestTools SaaSTesting SoftwarePerformanceAssuranceConfidential

  40. TraditionalandCloud-basedTestTools Cloud-based TestTools TraditionalTestTools QTP,WinRunner RationalRobot RationalFunctionalTesterSilkTest Functional GomezPushToTest,SOASTA Selenium,STAFF,Fitness,Watir CloudTesting LoadRunner,RationalRobot, RationalPerformanceTesterSilkPerformer uTest Manyothers Performance Jmeter,Grinder,WebLoad, OpenSTA SoftwarePerformanceAssuranceConfidential

  41. TraditionalVs.Cloud-basedTools TestingSaaSUsingTraditionalTools Manyproviders whoprovide SaaStesting usingtraditionaltestingtools Manyopensourcematuredtools Manyoptionsfor professional servicesbyindependent serviceproviders Professionalserviceoptions allowcheapercost TestingSaaSUsing Cloud-basedTools Afew providers whoprovide SaaStesting usingtheirown cloud-basedtesttools Limitedopen sourcetest tools options Captiveprofessionalservicebytest toolsprovider whichisnot evencore for theirbusiness Captiveprofessionserviceis soldat veryhighrate SoftwarePerformanceAssuranceConfidential

  42. Conclusion SoftwarePerformanceAssuranceConfidential

  43. Conclusion Customersbuyeachmonth–keepthemhappy SaaSis onlya deliveringmodel,customerswillnotcompromiseon functionality,stability,security,andperformance.Theywillnotswitchto SaaSit ifnotbetter thantraditionalsystems. Save timeonsupport,maintenance,andupgrade,etc. Youneedto pay moreattentionto stuff likeaccessibility,security, deployments.Thewayyouearnreputation Don‟tbuy cloud-basedtest solutionjustbecauseyouaretestingSaaS.In mostcases,traditionaltestsolutionsworkjustfineandsaveyour30%– 50% cost. SoftwarePerformanceAssuranceConfidential

More Related