200 likes | 367 Views
Using Atlassian Products in an Enterprise Environment. Mark McCormack British Telecom Inlife Support Lead Development Services Organisation. Outline. Using Atlassian products in an Enterprise Atlassian Product adoption Size of Confluence and Jira in our Enterprise
E N D
Using Atlassian Products inan Enterprise Environment Mark McCormackBritish TelecomInlife Support LeadDevelopment Services Organisation
Outline • Using Atlassian products in an Enterprise • Atlassian Product adoption • Size of Confluence and Jira in our Enterprise • Some Atlassian Use Cases • Customisations • Application Monitoring • Platform • What’s Next • Product Wishlist • Some Lessons Learnt • Q & A
About • Network is available in 173 countries. • Around 3,000 new customer sites every month • Carry 80% of all credit card transactions in the UK. • More than 90% of the UK’s financial institutions use our secure data networks • Rolling out super-fast broadband to 40% of the UK by 2012. • FTTC downloads of 40Mb/s and uploads of 2 Mb/s. • FTTP downloads of 100 Mb/s London 2012 Olympics
Development Services Organisation • We advise on best practices for all tools • Inlife team looks after all hosted applications: • We are an Atlassian shop • Subversion, Jive Forums, Nexus • Other in-house tools • Authz, Teams, IPPI, Provisioning, Static Data & PALMS • SDK Catalogue (will soon become BT Enterprise Repository)
Product History Dec 2003 • 5 people created a team (called code red) but there was no way of coordinating their efforts • Bought Confluence and Jira and installed it on a PC and put it in a meeting room June 2004 • Usage spread to a few development teams (200 users) • Bought new hardware and installed it in a server room (engineering.intra.bt.com) April 2006 • Another group installed a separate instance of Jira and Confluence to manage service improvements February 2007 • Management Services and Engineering decided joining forces would be a good idea • Userbase reached 300 and 2000 respectively • http://collaborate.intra.bt.com was born and hosted by VS Group on VMWare. July 2007 • A Virtual Administrator crew formed out of 35 volunteers from the userbase to perform end user support and administer the platform • Two users in particular did most of the Admin work. Userbase grows to over 9000 people.
Product Maturity August 2007 • Sponsors found to pay for new hardware & uplift plan created • Team split started to disrupt progress to achieve this and ownership was contested September 2007 • Userbase grows to 10,000 October 2007 • I/O Capacity of the disks in the VSP environment started to be exceeded. • Platform started to have outages with huge traffic increase January 2009: • Moved to an enterprise hardware infrastructure, Oracle 10G RAC & 24x7 support • Collaborate Wiki was formed, Confluence 2.10 and integrated with Crowd February 2009: • Introduced Storm & Jira Issues July 2009: • Started to assimilate other wikis (like twiki, rdrwiki, ProfComms)
Confluence Numbers • 800,000 pages • 102,000 attachments • 3000 spaces • 25,000 display requests per day • 30,000 actions per day • (eg login, edit, comments, rss feeds etc)
Numbers • 3 flavours of Jira • ACF – Architecture Conformance Framework • Issue Tracking (100 projects) • Story Management Operating Model • BT is adding > 20,000 stories every month
Confluence Use Cases • Tool Guides on how to get started • Tool Access Request Forms • We have a teams application we encourage before requesting a Space • Early stages of a Provision Application • Application Deployment Schedule • Collaboration between Dev and Inlife Teams (geographically dispersed) • Don’t send emails but rather send Confluence links • Trying to have an FAQ before raising an issue for help • Application Usage Statistics • MyBT has a dashboard interface and makes use of the Confluence remote API and render as html inside a portlet. • Most of the Ajax related actions don’t work
Confluence Customisations • Implemented the functionality of security classification requirements in the scaffolding plugin. • To implement this functionality the edit page / do edit page actions of the confluence needs to be overridden. Scaffolding plugin was already overriding the edit actions of confluence. • Confluence doesn't allow two plugins to override a particular action. • Extended the edit actions to include classification
More Confluence Customisations • https://collaborate.bt.com/staticdataplugin to provide a LOB picker from LDAP • Static Data Macro that retrieves platforms • BT Space Admin Confluence plugin makes use of our eorg DB in order to get the Line of Business functionality associated with a Space • Modified the Excel Plugin as it was hanging Confluence a lot • We had one user with a page that had 48 references to the excel macro • Every reference was an API call that forced a GC, so we disabled explicit GC • Page went from 14 minutes to load to about 4 seconds
Customisations • Hierarchy plugin • Static Data plugin to strip LOB info out from LDAP • Loads of constraints on Story creation and workflow • All Stories must have at least one child and at least on Acceptance Criteria • Organogram • Added Active User Report under Users, Groups and Roles • Roles for Storm • Loads of different workflows (put issues On Hold whilst with Dev team or Vendor)
More Customisations • https://collaborate.bt.com/fisheye plugin to allow the authorisation of SVN repositories (handled by hasPermissionToAccess method which is overrided to authorise against our Authz). • Use https://collaborate.bt.com/crowd client libraries for apps like https://collaborate.bt.com/forum • We have a Jira Project that helps us keeps track of Subversion Projects ensuring everyone has a Salsa ID. • Unified Rolling Calendar Plugin
Application Monitoring • ZXTM • Nagios alerts via email for slow (or dead) apps. • JMX is enabled for most of our java apps • Disk space (e.g shell script to alert > 80%) • Backup successes or failures uploaded to our wiki • Access logs with IP tracking • Usage statistics to figure out what our users are doing • Perl script we run for snapshots of ugly apps • YourKit Profiling from within IntelliJ • kSar
Platform • Currently 18 HP Blades mostly BL460s with 2 Quad cores and 32GB of RAM for our existing hosted application servers • RHEL 5 (soon OEL) • Were using Weblogic 10.3 but now Tomcat 6 • Java 6.0.18 • ZXTM 5.0 • Oracle 10g in a Cluster hosted by another team on HP Blade 680s with 2 Quad cores and 64GB of RAM • Crowd is our only clustered application (currently) • We use Capistrano to help automate our deployments and allows easy rollback
What’s Next? • Clustering of Confluence, Jira and SVN • Triple our hardware for Disaster Recovery • Archiving & stats analysis plugins (customised) • Improve provisioning • All the Atlassian 2s (Bamboo, Crowd, Fisheye, Crucible) although upgrading to Fisheye 2.0 will be expensive on disk space. • Start charting long-term trends with better monitoring • Assimilate all wikis (like LimsWiki for one)
Wishlist Confluence Wishlist • Tighter integration with Jira • Allow Jira Issues to be raised via Confluence and vice versa • Allow profiles to have attributes (like team, group or project) • Confluence attachment storage API. We want to store attachments away from Confluence and write our own adaptors to do this. • Improve the edit permissions cascading security • Customisations of actions (eg. Overrides to allow > 1 plugin) Jira Wishlist • Allow two columns for attributes • Allow picklists to show more values without scrolling • Smart forms that can include wiki macros • How can we hide Administrators from assignee list? • Can Greenhopper scale? How compatible is it with existing 3.13.2 plugins/customisations?
More Wishes Jira … • Would like to use our Issues as a source for FAQ before users raise an issue? • Ought to be Confluence tightly coupled with Jira Issue labels • Timebooking to allow resource Management? Others • Fisheye 2 IE6 browser support • Make Crucible a viable app for the enterprise (we have some open Atlassian issues for 10k+ users) • Can we turn on “remember my login on this computer” checkbox during login?
Lessons Learnt • Be wary of any plugin • Excel Macro plugin • We turned off the usage tracking plugin • We trialled the stats analysis plugin but its currently off since we got hammered two lunchtimes in a row shortly after going live • WebLogic seems to be a lot slower than tomcat • Confluence 3.x is much better than 2.x • We had problems with Crowd 1.6 client and currently use Crowd 1.3 client • Tune the JVM heap • Confluence: we use java –server –Xms:8G –Xmx:8G PermSize=512m MaxPermSize=512m +UseParallelGC +AggressiveHeapLargePageSizeInBytes=4M -DisableExplicitGC • Jira: we use –server –Xms4G –Xmx4G PermSize=1G MaxPermSize=1G +UseParallelOldGC +AggressiveOpts • Crowd: we use java –server –Xms:4G –Xmx:4G PermSize=512m MaxPermSize=512m • Fisheye: not really tuned yet –Xms:4G –Xmx:4G MaxPermSize=512m
Questions mark.j.mccormack@bt.com British TelecomInlife Support LeadDevelopment Services Organisation