510 likes | 590 Views
GrowZone Online …. The Challenge to Deliver Internet Services to Southern Inland Queensland. Tony Nugent & Phillip Gersekowski. Project Background …. Community owned and managed GrowZone Development Network Inc Up to 44 IMC groups in the community Regional Economic Development Focus
E N D
GrowZone Online … The Challenge to Deliver InternetServices to Southern InlandQueensland Tony Nugent & Phillip Gersekowski
Project Background … • Community owned and managed • GrowZone Development Network Inc • Up to 44 IMC groups in the community • Regional Economic Development Focus • Planning & Deployment supported by • Networking the Nation Grant • Will operate commercially with … • Full ISP Services • Local Public Access Terminals (Internet Cafes)
The region … • 29 Local Authorities • 412,000 klms2 • Population: 226,667
Project Goals … • Internet access for the cost of a local call • Up to 44 POPs in the GrowZone region • Call-back to subscribers on Pastoral or STD calls • Public Access Terminals in each community • 3 Computers, 2 printers, scanner and video camera • Access to local and regional information • Network infrastructure for a Regional Communication and Information Network or Web-based database • Locally owned, managed, updated and accessed
Additional Project Benefits … • Enhanced Business-to-business communication • Fast, reliable LANs, WANs and Internet Connectivity • Locally Managed – increased IT skills base • NDC – team of 3 • Software – team of 2 • HelpDesk – team of 3 • Regionally – up to 30 local IT people and trainers • Regional IT training platform • Unix/Linux Systems Administration & Network Management • Web-based interface SQL Development • Encourage THUUG activities • Work Experience opportunities throughout the region – CJP application = 15 jobs
Network Hardware … • Toowoomba Server Farm • 2Mb Fibre Megalink (Telstra) • 8 High-end PC Servers running Linux • Multi-homed Linux routers/firewalls • Points Of Presence • Digital OnRamp 2 and Frame Relay links to network • PC Servers running Linux • Cisco Routers and/or Rastel cards • Public Access Terminals • PC Units running Windows 98 • Linux Server running Samba (netbios) network services
Network Design … Low density population Long distances Many small isolated communities Computer illiterate
Software … • Basic Operating System: • Redhat 6.x • Network Server Services: • DNS, WWW, FTP, Mail, Proxy, News • NFS, Samba, NIS, DHCPD, MySQL • Backup, Network Monitoring & Management • User Management & Billing: • Customized PHP3, PERL, MySQL • Routing & Dial-up Support: • Radius, PortSlave, PPP
Our First Challenge … • A Few People, Long Distances and A Really Old Telephone service … • Pricing the service so it pays for itself:“$2.85/hour … that’s dearer than …” • Delivering a reliable dial-up service:“I keep getting disconnected …” • Contain Network Running Costs:“Our ISDN Costs are killing us …” • Managing Remote Sites cost-effectively:“I’ll be there in 9 hours …” • Solving “Last-Mile” Problems:“11,200 bps … that’s great … thanks!”
The Next Challenge… • DRCS… Digital Radio Concentrator System • Radio/UHF Phone System • Work on a Concentrator System • Can be multiple hops from an end user until the Terrestrial Telco Network is reached • Maximum achievable Connect Speeds – 12,000 bps • Normal speed vary from 2400 bps to 7200 bps • Minimum speeds as low as 300bps • Problems usually alleviated with specialized Modem Initialization Strings…. • Different based on modem chipset • Telstra Upgrading all DRCS in Australia to HCRC – High Capacity Radio Concentrator • Maximum Speed on HRCR up to 31,200 bps • Average speeds 19,200 bps to 28,800 bps
Other Challenges…. • How to provide local call access to all subscribers within the region • Most subscribers live within local call to one of our POPs • Still….. A large proportion of subscibers are in extended zones • STD Rates $5 - $14 per hour at peak times • Pastoral Calls – 20 cents per 5 Minutes - $2.40 per hour
Telstra Dial-Connect…… • Perfect solution to our problems • Local call cost to the subscriber • Designed for IP based data communictions • Except…. • Highly Expensive • Average Cost for our client base - $6 - $10 per hour charged to the Business (GrowZone) • Difficult to manage and control • Cannot force people within the local call zones to our POPs to use the local POP • Service could be exploited to the financial detriment of our project • Ease way to go broke…… • $2.85 per Hour revenue vs $8 per Hour Average Servicing Cost
Solution…… NT PPP cbcp • Microsoft sometimes does some good things… • NT Servers support Callback Control Protocol (cbcp) to allow more secure RAS connection • Authentication mechanism means that a user is called back to verfiy there request for access. • Problem – NT has the only full Server Side CBCP Support • Solution…implement cbcp server support for the Linux PPP daemon… • Because we have the source code we can create a new feature in pppd • Additionally we have to integrate Radius Support in pppd to cated for cbcp support….. • Patch the Open Source Radius Server to provide Radius Attributes for cbcp support…
Results… • We can now provide local call access to any Rastel POP via cbcp support in pppd, that is controlled by Radius… • Cost of providing local call access drops from $8-$10 per hour to $2.40 per hour • Financially manageable with a $2.85 per hour retail pricing. • Continuing Development to assist in alleviation of problems with poor quality phone Lines in extended Zones (DRCS) • Add features to Radius and cbcp support to force harsher modem initialization parameters before commencing the callback
Other Challenges…cont. • Provide High Speed Internet Access to PAT (Public Access Terminals) in location that are local call to existing GrowZone OnLine POP’s • OnRamp2 Express Plans for WAN Connectivity are compartively expensive compared to PSTN Services. • Good Quality PSTN Services are almost equivalent to ISDN Services • 64 kbps to 56 kbps (realistically 44 kbps) • Wouldn’t it be good to use multiple 56 Kbps PSTN Dial-Up’s and Channel Bond them together – Multi-Link PPP ? • Linux pppd does not yet support MLPPP • Alpha Version MLPPP is good but unstable with line drop-outs…
Linux/Cisco to the Rescue • Features of the Late 2.2.x Version Linux Kernels provide strong support for load balancing over Serial Links (PPP) • Cisco ISO Implementation provide load balancing over multiple paths through Process Routing. • Using these two features, the fastest Dial-Up Internet Access in Southern Queensland is in Oakey • 4x 56 kbps to Toowoomba yields approx 25 Kbytes/sec Internet Access • Approximately 176-256 Kbps bandwidth to Oakey • Currently used in 4 Sites • Reduces WAN Costs by approximately $300 per site per month….
Future Challenges … • Largely inexperienced Computer users … • Overcoming Community Resistance:“The Internet … no way … we don’t want it …” • Overcoming Techno-phobia:“No mate … never turned one on before …” • Overcoming Sheer Bloody Mindedness:“I put the disc in and nothing happened …” • Finding Good IT People: “No-one out here knows anything …”
Software Development….. • Open Source Software…… • Allows Changes to be easily made…. • Modifications to Services to add features, improve security, and develop new products • Radius Servers • PPP implementations • Callback • Solve poor client side implementations • Enhance Features – Termination Cause
Software Development….. • Determine Causes of Problems….. • Look at the Source Code…… • Control • Determine our own destiny • Vendor and Support Provider independence • Allows for quick turn around on problem diagnosis and solutions
In-House Development • Specialized User Management System • Add/Remove/Modify Users • Update Unix System Files • Create Email Addresses • Build Skeleton Home Directory • Virtual ISP User Management System • Access Controls • Hides all other users within the system from the VISP Owner/Manager
In-House Development • Automated Billing and Reporting • Billing users access to multiple accounts • VISP Master Account • User Account • Format Invoices differently for different users • Calculate Commissions for IMC Payments
Network Statistics….. • May’s Statistics • 39 Gb of Data per Month • 22,000 Hours of Access • 22,000 Individual Connections per Month • Approximately 1000 Users • Network Line Rentals • Approximately $40 K per month • Bandwidth Costs • Currently 10% of Line Rentals • Expected 25-30% of Line Rentals
More Stats….. • 40+ Remote Servers • 50+ Remote routers • 400+ Dial-Up Access Lines • 6 Frame-Relay Services • 23 ISDN Services
It’s a large network….. • Measure of the Network’s size is geographic not technical (Traffic and Bandwidth)
Proxy Servers • All HTTP Traffic (port 80) is transparently proxy cached at each node in the Network • 60 – 80% of all traffic is HTTP • Proxy Server achieves 24% Hits on all requests • Proxy Server saves 8% of Volume Costs based on Hits ratios (due to increasing dynamic content on the internet)
Content Management… • Open Source Proxy Server allows for development of third party extensions • Content Management – Filtered Content is supplied to all PAT’s • Bans Adult Content, Drug Related Information, Violence and Aggressive Sites, etc… • Controlled by lists of sites that can be gathered from the internet. • Able to scan the Cache Files for the Proxy Server for all users using the network, to increase the size and coverage of the filtering lists.
Remote Network Management • How to Manage 44 Remote Servers and Routers….. • Unix to the Rescue • telnet • ssh – simple and Secure • nfs • Network Management Tools • netSaint – SMS and Email Alerts • mrtg
Remote Workstation Management • How to manage, maintain and support 120+ remote workstations • SOE – Standard Operating Environment • Workstation OS Installations are expendable • any problems simply crash and burn and re-ghost • VNC • Ghost • PXE – Network Boot Agents/Network Boot ROMS • Samba
Power Management • Nightmare……….. • Servers do not like power disruptions • Each Server and Router is connected to a UPS • UPS Management Software control graceful shut down of servers and restarts server when power is restored • Allows for Remote Monitoring of UPS Statistics and Functioning
Technicians at Remote Sites • Definitely not……. • Most Sites are run with community volunteers • Some Sites run by the owner of the Local Store, a librarian, of staff of the business in which the Site is located
War Stories…….. • Can you reboot the Router…… • What is that…. • A small black box with Cisco written on it about the size of half a pizza box….. • Do you Mean the thing with all the cables coming out of it with IBM written on it… • No the small black box with Cisco Written on it – it will have 4 Cables plugged – all of different colors • Do you mean the thing that is connected to the Printers with LexMark written on it…. • No the other black box – that last on in the place • Oh the one with Cisco Written on it – Yes I have found it what do I do now • Turn the Power Off and then back On….. • How do I do that……
….Technicians at Remote Sites • All Site Management Must be performed remotely • We cannot rely on there being anybody on site (physically or mentally) • Each Site is designed with multiple Back Door Entrances • Via WAN Connection – Frame Relay, ISDN, or PSTN • Via Dial-Up Ports – ISDN or PSTN • Via additional Management Dial-In Modem
Security must be Maintained… • Each Backdoor is secured to restrict access…… • No direct telnet or ssh access from any backdoor – must hop across each router in the network and authenticate separately to each….
Ongoing Network Management… • Measure Bandwidth and Dial-Up Port Utilization • Change the structure and layout of the network over time to improve performance and/or reduce costs • Network Rationalization …“Doing more with less …” • Re-design Network Design • Lower Telco Costs • affect on Bandwidth • Using Alternative Techniques • Call-back services – lowering Call Costs • Multi-link / Load Balanced PSTN PPP Connections