Revisiting APAN Services #2

1. Revisiting APAN Services #2 Yoshikata Hattori, hattori@noc.kddnet.ad.jp Pensri A., pensri@cs.ait.ac.th Lee, Jaehwa, jhlee@noc.kr.apan.net APAN NOC 19th APAN Meeting, Bangkok

2. What Are APAN Services? • WWW • apan.net and www.apan.net • DNS • ns.kaist.apan.net and ns.jp.apan.net • E-mail/mailing lists • apan.net • Distributed among/operated by APAN-KR/ANF and APAN-JP NOC’s

3. Why Revisiting? • These are the most important services for us • to get information from APAN thru WWW • to communicate with others thru e-mail/mailing lists • based on the APAN DNS • So they need • correctness of information • reliability and stability of operation/monitoring • And they are naturally based upon the network architecture/operation. • Now APAN network architecture/operation has changed greatly which requires revisiting the services. • 24x7 operation/monitoring • GbE connection between JP and KR

4. (Previous) Problems • WWW • Contents of apan.net(KR) and www.apan.net(JP) have 4 hours’ difference -> Harmful • DNS • No backup of primary database(KR) -> Dangerous • E-mail/mailing lists • No backup of mailing lists(KR) -> Dangerous • Operated/monitored jointly by APAN-KR/ANF and APAN-JP NOC’s • No 24x7 operation/monitoring on KR side

5. New Scheme • Servers distributed among JP and KR • Controlled/operated/monitored by APAN NOC • Redundancy/reliability • Information correctness, reliability, and stability • NFS between servers for WWW • Backup of data for WWW, DNS, Mailing Lists • Servers location independence of the sec.

6. Current Status/Follow-up • WWW servers, apan.net = www.apan.net • 2 official servers(JP and KR) with 1 hidden server(master.apan.net in Sec./TH) • Sec controls the contents • Hidden server is rcync’ed by JP server (with a reliable backup) in every 4 hours • Sec must have a way to trigger rsync • KR server NFS-mounting JP server contents • KR must have a local copy : local copy of NFS-mounted contents • Need performance test for this scheme • DNS servers • Primary server moved to APAN NOC from KAIST, but it’s hidden now • The same 2 servers(secondary) seen from outside • 1 hidden server + 2 servers or just 2 servers? • Mail server/mailing lists reconfiguration • Still pending • Should follow WWW servers scheme – 2 official mail exchangers • Sec must control ML lists • Is it worth trying anycast for these services?

7. Current Status on KR Side • KOREN/APAN-KR NOC has moved to Seoul with servers • I(JH Lee) am working for Convergence Lab., KT in Seoul • Our new servers (still going on) • 2 redundant 1-u servers for WWW, DNS, mail servers w/ storage servers • These will host the APAN servers/services • Planning to have specialized servers for tunnel broker, AG bridge servers, SNMP servers, etc. • Only in 6 years we’re going to have many new servers

8. Figure of APAN WebServers Relocationby Mr.Hattori Domain Name Servers of apan.net TH JP KR apan.net A 203.181.248.30 A 203.255.255.86www CNAME apan.net. Secretariats can edit and update web contents on master.apan.net. Slave:203.181.248.3 Master:203.255.248.57 These A records and CNAME record realize round robin service. JHLee-san sent CD-Rs to Pensri-san. They contain the whole web contents of apan.net. And Pensri-san has uploaded them on master.apan.net. Master:192.249.24.62 WebContents Old KR web server Previous rsync configuration between old KR server and JP had deleted. WebContents WebContents WebContents Mounted with NFSReal-time updating can be done Synchronizing the contents by SSH-wrapped rsync every 4 hours master.apan.net203.159.31.33 ns2.jp.apan.net= apan.net= www.apan.net203.181.248.30 noc6-5.kr.apan.net= apan.net= www.apan.net203.255.255.86 $ cat rsyncd.confhosts allow = 203.181.248.30use chroot = nomax connections = 4syslog facility = local5# pid file = /var/run/rsyncd.pidtimeout = 6000[www] path = /usr/local/src/www/html/apan.net lock file = /home/inetapan/rsyncd.lock uid = inetapan gid = users read only = true Users can access JP or KR server using http://apan.net/ or http://www.apan.net/.The result of DNS query determines which server will be selected. %crontab –l20 */4 * * * /usr/home/httpd/cron/wwwsync/wwwsync.sh%cat /usr/home/httpd/cron/wwwsync/wwwsync.sh#!/bin/sh/usr/local/bin/rsync -e ssh -aqz inetapan@master.apan.net::www /home/httpd/www.apan.net Results of DNS query are round robin. 1st time %nslookup apan.net Name: apan.net Addresses: 203.181.248.30, 203.255.255.86 2nd time %nslookup apan.net Name: apan.net Addresses: 203.255.255.86, 203.181.248.30 3rd time %nslookup apan.net Name: apan.net Addresses: 203.181.248.30, 203.255.255.86 This crontab with script on JP server remotely runs rsyncd command wrapped by SSH every 4 hours. Then rsync checks the updated contents on master.apan.net and transfers them to JP server. This rsyncd.conf on master.apan.net allows rsync accessing from JP server.

9. Redundancy for Web Service • How to build redundancy for http://apan.net/ and http://www.apan.net/ • Synchronize contents from TH to JP and from JP to KR • Allocate 2 IP addresses (KR:203.255.255.86 and JP:203.181.248.30) for apan.net and www.apan.net • Use round robin DNS • How to synchronize the web contents • The bandwidth and RTT of TH-JP and KR-JP are taken into account • KR-JP use NFS, enough bandwidth and good RTT • TH-JP use SSH-wrapped rsync because of limited bandwidth

10. Building KR-JP Synchronization by NFS • NFS for synchronization between KR and JP, and he led the implementation • NFS has already showed enough performance within Korea • Fortunately, there is enough bandwidth between KR and JP • JP server, exports the web contents as read-only NFS server only to KR server • KR server remotely mounted them as NFS client • Destination is from JP to KR • Need further tests for NFS/WWW performance

11. New Services • NTP • Information/Routing Registry • H.323/SIP • APAN Observatory • LDAP • … • Any services members want to have

12. Now comes the detailed report of the APAN services relocation by APAN/APAN-JP NOC