1 / 51

Chicago, IL

®. C20. Using Multicast in real life scenarios. Murtuza Choilawala – Global Response Team (GRT). Chicago, IL. April 24-27, 2006. Using Multicast in real life scenarios. Presented by Murtuza Choilawala - GRT. Agenda. Introduction Multicast Distribution Overview

ryanadan
Download Presentation

Chicago, IL

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. ® C20 Using Multicast in real life scenarios Murtuza Choilawala – Global Response Team (GRT) Chicago, IL April 24-27, 2006

  2. Using Multicast in real life scenarios Presented by Murtuza Choilawala - GRT

  3. Agenda • Introduction • Multicast Distribution Overview • Multicast Log Files and Commands • Configuring and Tuning Multicast • Multicast Troubleshooting • Multicast Performance Tuning • Multicast Performance Evaluation • Conclusion

  4. Introduction • Tivoli Multicast became available with • Tivoli Management Framework v4.1.x • Tivoli Configuration Manager v4.2.x • Multicast data distribution can be a powerful solution when distributing bulk data to numerous targets in a large-scale environment • There are some considerations in multicast implementation • To make multicast stable and reliable • To improve multicast performance • Multicast very much depends on the environment such as network or background traffic • Some customer’s parameter settings may not work for other customers • Performance tuning approach will be different for each environment • Each customer has to find the best configuration for their environment

  5. Multicast Distribution Overview

  6. Multicast Distribution Overview • Preparation Phase: • Step 1: Prepare Files to be Distributed • Step 2: Create Software Package • Data Distribution Phase: • Step 3: Distribute Software Package • Step 4: Resend Dropped Packets • Installation Phase: • Step 5: Install Software Package

  7. 1 RMTPOpenReceiver() 3 RMTPOpenSender() 2 RMTPConnectReceiver() CONN RMTPConnectSender() 4 CACK RMTPAcceptConnection() 5 Data Xfr RMTPSendData() 6 RMTPReceiveData() 7 POLL NACK 8 Data Xfr ACK 9 RELEASE RACK RMTPClose() RMTPClose() Multicast Distribution Overview • Data Distribution Phase (CONN – DT – REL)

  8. Multicast Log File and Commands • To perform multicast troubleshooting or performance tuning, multicast log file and command output information are mandatory • Log files: Multicast distribution details • Command outputs: Multicast configurations, parameters, and distribution status • It is possible to gather multicast log files and command output by using scripts. • The script will enable you to automate the process of collecting log files and command outputs for each distribution • Need to understand which log files and command outputs are required for multicast troubleshooting and performance tuning

  9. Multicast Log Files • mcast.log File • Most important log file for Multicast troubleshooting and performance tuning • Contains communication flow and detailed data transmission log information • Located at $DBDIR/mcast.log on repeaters • Debug level = 3 is recommended when tuning or troubleshooting multicast (default = 1) • lcfd.log File • Contains detailed data transmission process on a specific endpoint • Contains endpoint information about endpoint methods that are invoked on a specific endpoint via downcall • Located at $LCF_DATDIR/lcfd.log on endpoints • Debug level = 3 is recommended when tuning or troubleshooting multicast (default = 1)

  10. Multicast Log Files • gatelog File • Contains detailed information of endpoint methods execution and endpoint login • Software package installation • Starting multicast receivers • Located at $DBDIR/gatelog on gateways • Debug level = 9 is recommended when tuning or troubleshooting multicast • Software Distribution Log/Trace Files • Can be used for software package installation troubleshooting • Log file is located at $BINDIR/../SWDIS/WORK/ • Trace file is located at $LCF_DATDIR/../../swdis/1/ on endpoints • Trace level = 5 is recommended when troubleshooting (default = 0) • epmgrlog File • Can be used for endpoint troubleshooting • Located at $DBDIR/epmgrlog on Tivoli Server

  11. Multicast Commands • wmcast Command • Can be used for configuring multicast parameters • wmcast –s <repeater> [keyword=value…] • Can be used for browsing current multicast configuration • wmcast –s <repeater> • Can be used for verifying connections (multicast ping) • wmcast –p {<repeater> | all}

  12. Multicast Commands • wmcast –s Command Output dtwtout: 3000 (ms) relwtout: 2000 (ms) connwtout: 2000 (ms) connrtry: 5 dtrtry: 10 pollrtry: 5 relrtry: 5 ttl: 5 blocksize: 1460 (Bytes) backofftm: 100 (ms) repeat: 2 rcvbufsz: 250000 (Bytes) sndbufsz: 250000 (Bytes) mc_netload: 500000 (Bytes/sec) recv_timeout: 1800000 (ms) port: 9499 returnIP: 0.0.0.0 mcadvert: 224.0.1.118 mchigh: 224.2.255.255 mclow: 224.2.128.0 ifsrcaddr: 0.0.0.0 ifrcvaddrs: 0.0.0.0 mc_debug_level: 1

  13. Multicast Commands • wmdist: Configuring Parameters • Can be used for repeater configuration • wmdist –s <repeater> [keyword=value…] • Make multicast available on a specific repeater with one of the following parameters: • repeater_multicast • endpoint_multicast • default_multicast • fail_unavailable • Can be used for browsing current repeater configuration • wmdist –s <repeater>

  14. Multicast Commands • wmdist –s Command Output repeater_id: 1554731128.1.577 rpt_dir: C:/Tivoli/db/ishii1.db/tmp/ permanent_storage: TRUE max_sessions_high: 5 max_sessions_medium: 10 max_sessions_low: 40 disk_max: 500 (MB) mem_max: 64 (MB) send_timeout: 300 (secs) execute_timeout: 600 (secs) notify_interval: 30 (mins) conn_retry_interval: 900 (secs) retry_ep_cutoff: 7200 (secs) net_load: 500 (KB/sec) packet_size: 16 (KB) target_netload: 0 (KB/sec) debug_level: 3 repeater_multicast: FALSE endpoint_multicast: FALSE default_multicast: FALSE SLOW_LINK: FALSE

  15. Multicast Commands • wmdist Command: Display Distribution Status • Can be used for displaying distribution status for each distribution • wmdist –l • Can be used for displaying distribution status for each target • wmdist –e <dist_id> • For more detailed distribution status: • wmdist –liv <dist_id>

  16. Multicast Commands • wmdist –l Command Output Name Distribution ID Targets Completed Successful Failed mcast_test^1 1978508757953160036 4 1( 25%) 0( 0%) 1( 25%) mcast_test^2 1757544609953651663 1 1(100%) 0( 0%) 1(100%) • wmdist –e <dist_id> Command Output Name Status Start Time End Time tivoli CANCELED 2003.02.06 20:11:50 2003.02.06 20:59:01 a24922 SUCCESSFUL 2003.02.06 20:39:59 2003.02.06 20:44:32 a24927 SUCCESSFUL 2003.02.06 20:39:58 2003.02.06 20:44:28 a25065 SUCCESSFUL 2003.02.06 20:40:00 2003.02.06 20:45:01 a25592 SUCCESSFUL 2003.02.06 20:40:00 2003.02.06 20:44:52 a25593 SUCCESSFUL 2003.02.06 20:39:58 2003.02.06 20:44:49 a25594 SUCCESSFUL 2003.02.06 20:40:00 2003.02.06 20:44:23 • The elapsed time includes software package installation process Elapsed Time = Data Transmission Process + Installation Process

  17. Configuring Multicast • Send a multicast ping (wmcast –p) to all targets with the default parameters • Ensure that multicast can be enabled between between sender and receiver • If some receivers do not respond to multicast ping, the receiver must have some problems STEP 1 Sending Multicast Ping Sending Data via Multicast STEP 2 • Send data via multicast with the default parameters • Check the distribution status with wmdist and look into the problems with the appropriate log files if the distribution failed Tuning Multicast Parameters • Ensure that data can be distributed via multicast without any errors • Tune multicast parameters and configurations to improve multicast distribution performance and throughput STEP 3

  18. Step1: Sending Multicast Ping • Data Stream of multicast ping between sender and receiver is the same as when transmitting data to the target via multicast • Multicast ping allows us to make sure whether multicast is enabled between sender and receiver • If multicast ping fails for a target, multicast data transmission to the target cannot be completed • Multicast ping is the first thing to do and should be exercised before sending data via multicast • wmcast –p a00028 Multicast ping was successful. a00068 Multicast ping was successful. a00069 Multicast ping was successful. a00062 Multicast ping was successful. a00009 Multicast ping was successful. a00062 Multicast ping was successful. a00010 Multicast ping failed. a00053 Multicast ping was successful.

  19. START wmcast –p Success? Yes No Check Multicast Configuration Check Receiver Status/Reachability Check Software Prerequisites Collect Multicast Log Files Check Miscellaneous Check Network Configuration GO TO STEP2 Step 1: Sending Multicast Ping • Recommended Checkpoints for Multicast Ping Problems

  20. Step 1: Sending Multicast Ping • If multicast ping fails on an endpoint, make sure of a couple of check points and take a look at log files and messages corresponding to the endpoint • Network and endpoint connectivity • mcast.log, lcfd.log file, etc. • Endpoint is unreachable • Receiver process is not running • Multicast is not enabled • TTL (Time-to-Live) configuration Typical multicast ping problemsM • Multicast advertisement setting for the gateway matches the endpoint <MCAST_ADVERTISEMENT> <MCAST_TEST_ADVT>MCAST_TEST_ADVT</MCAST_TEST_ADVT> <SENDER>1808441727.1</SENDER> <PACKAGE_ID>testAdvert</PACKAGE_ID> <PACKAGE_VERSION>testAdvert</PACKAGE_VERSION> <DISTRIBUTION_ID>testAdvert</DISTRIBUTION_ID></MCAST_ADVERTISEMENT>

  21. 1 RMTPOpenReceiver() 3 RMTPOpenSender() 2 RMTPConnectReceiver() RMTPConnectSender() 4 RMTPAcceptConnection() 5 RMTPSendData() 6 RMTPReceiveData() 7 8 9 RMTPClose() RMTPClose() Step 1: Sending Multicast Ping Multicast Data Transmission Flow CONN connwtout * connrtry CACK Data Xfr pollwtout * pollrtry POLL NACK dtwtout * dtrtry Data Xfr ACK relwtout * relrtry RELEASE RACK

  22. Step 1: Sending Multicast Ping • Multicast Data Transmission (Multicast Ping) Flow: Log Messages in lcfd.log File Feb 04 18:25:35 3 RMTP 64 Receive CONN packet from [10.94.142.128:39513] ackcnt[0] Feb 04 18:25:35 3 RMTP 64 Set back-off time[9] - range[100] Feb 04 18:25:35 3 RMTP 64 << CLOSEDCONN -------------------------------------->> Feb 04 18:25:35 3 RMTP 64 Datasize[289] Msgsize[478405499] Msgnum[1] MsgID[0] Blocksize[7300] Blocknum[1] Feb 04 18:25:35 3 RMTP 64 Set destination port[39513] connection_id[44860] Feb 04 18:25:35 2 MCASTR 64 Entering decrypt_in_place... Feb 04 18:25:35 2 MCASTR 64 Leaving decrypt_in_place, rc = 0. Feb 04 18:25:35 3 MCASTR 64 Advertisement After ConnectReceiver: Feb 04 18:25:35 3 MCASTR 64 RMTP version: 4 Datasize: 289 Feb 04 18:25:35 3 MCASTR 64 Optional Data: <?xml version="1.0" ?> <MCAST_ADVERTISEMENT> <MCAST_TEST_ADVT>MCAST_TEST_ADVT</MCAST_TEST_ADVT> <SENDER>1808441727.1</SENDER> <PACKAGE_ID>testAdvert</PACKAGE_ID> <PACKAGE_VERSION>testAdvert</PACKAGE_VERSION> <DISTRIBUTION_ID>testAdvert</DISTRIBUTION_ID></MCAST_ADVERTISEMENT> • Multicast (including multicast ping) exchanges XML format data that contains information of the distribution data in the same manner as actual data transmission flow (CONN – DT – REL) before transmitting actual data (i.e. software package). This process is called multicast advertisement and multicast ping also performs multicast advertisement at the beginning of the distribution

  23. Feb 04 18:25:45 3 RMTP 64 Receive DT packet from [10.94.142.128:39513] Feb 04 18:25:45 3 RMTP 64 Receive POLL[1007] packet from [10.94.142.128:39513] cyclecnt[1] ackcnt[5] Feb 04 18:25:45 3 RMTP 64 Send ACK[1002] to [10.94.142.128:39513] • Sender sent DT (Data), and then receiver received POLL from the sender. The entire data was received and the receiver sent ACK to the sender Feb 04 18:25:35 3 RMTP 64 Receive CONN packet from [10.94.142.128:39513] ackcnt[0] Feb 04 18:25:35 3 RMTP 64 Receive CONN packet from [10.94.142.128:39513] ackcnt[0] Feb 04 18:25:35 3 RMTP 64 Send CACK[1009](YES) to [10.94.142.128:39513] • Receiver received CONN and then the receiver sent CACK to the sender as response Step 1: Sending Multicast Ping Feb 04 18:25:57 3 RMTP 64 Receive REL packet from [10.94.142.128:39513] Feb 04 18:25:57 3 RMTP 64 Receive REL packet from [10.94.142.128:39513] Feb 04 18:25:57 3 RMTP 64 Send RACK[100b] to [10.94.142.128:39513] Feb 04 18:25:57 3 RMTP 64 << CLOSED ------------------------------------------>> • Receiver received REL from sender. As a result, the receiver sent RACK and the multicast data transmission (multicast ping) was completed

  24. Step 2: Sending Data via Multicast • If Step 1 is completed, send actual data via multicast • Start with the default parameters • To perform in-depth analysis, send a large data (i.e. 500MB or larger) • To make log analysis efficient, minimize log messages • Disable ‘Retry Unicast’ option • Stop receiver process on targets that you do not want to send data • Check the distribution status with wmdist –e <dist_id> command • If distribution fails • To analyze the distribution and detect the problems, gather appropriate log information • Check typical multicast distribution problems • If distribution is successfully completed • To tune multicast distribution performance, modify appropriate parameters and check the results

  25. Step 2: Sending Data via Multicast • Recommended Checkpoints for Multicast Data Distribution Problems START Multicast Data Distribution Success? Yes No Check ADVT_TIMEOUT Error Check ABORT Error Check execute_timeout Error Set backofftm Properly Check Miscellaneous GO TO STEP3

  26. Step 3: Tuning Multicast Parameters • Once multicast becomes stable, multicast performance tuning needs to be done • Multicast performance tuning will very much depend on environment; so there is no standard • Starting with mc_netload parameter tuning is recommended • Key points in multicast performance tuning • Environmental Characteristics: OS, Number of targets, Distribution requirements, The other applications running on the same box or network, etc. • Network Characteristics: Type of network (WAN, LAN, etc.), Network bandwidth, Background traffic, etc. • Distribution Characteristics: Data size, Distribution frequency, Distribution time (day or night), Typical distribution, etc.

  27. Step 3: Tuning Multicast Parameters START • Recommended Checkpoints for Multicast Performance Tuning Tune mc_netload Any Large Software Package? No Distribute via Satellite or WAN? No Yes Check Depot and Staging Config Yes Tune blocksize and messagesize (*) Tune Multicast Buffer More than 50 Endpoint Targets? No Tune Miscellaneous Configurations Yes Tune OS and Network Increase max_sessions END

  28. Multicast Troubleshooting • Typical multicast distribution problems • Multicast timeout (ABORT) error • Wait timeout (ADVT_TIMEOUT) error • Execute_timeout error • Time-to-Live (TTL) count • The bottom line is to make multicast distribution successful distribute to all targets and make multicast distribution stable

  29. Multicast Timeout (ABORT) Error • Multicast timeout (ABORT) error is one of the most typical problems in multicast data distribution • Especially in a large-scale environment • ABORT error will occur when • Multicast dropped a lot of packets during communication • Multicast reached its timeout • A network problem occurred during communication • Receiver cannot hear sender, or sender cannot hear receiver during multicast distribution • As a result multicast reaches timeout on the sender side

  30. Multicast Timeout (ABORT) Error How ABORT error occurs during multicast distribution • Multicast communication starts • Sender sends CONN to receiver • Sender does not receive any response from receiver • Sender retries sending CONN several times (defined in parameters) • Sender cannot receive a response from receiver until it reaches timeout (connrty * connwtout) • Sender sends an ABORT packet • When receiver sees the ABORT packet, receiver quits listening and the multicast communication between sender and receiver terminates

  31. Multicast Timeout (ABORT) Error • ABORT error usually occurs in connection phase (CONN – CACK) or advertisement phase • During multicast connection phase, all receivers were sending back CACK’s to the sender within a short period and it dropped a lot of packets • If ABORT error occurs, you will see log message • mcast.log file • lcfd.log file Feb 04 20:00:39 3 RMTP 257 Receive ABORT[1006] packet from [10.94.142.128:42018] nameid[1808441727(10.161.202.107) 174] Feb 04 20:00:39 3 RMTP 257 << CLOSED ------------------------------------------>>

  32. Multicast Timeout (ABORT) Error • To resolve ABORT error, the following parameters need to be tuned • backofftm • wmcast –s <repeater_name> backofftm=time • Timeout (connwtout, pollwtout, dtwtout, and relwtout) • Retry (connrtry, pollrtry, dtrtry, and relrtry) • wmcast –s <repeater_name> connwtout=time • Especially backofftm, connwtout and connrtry parameters are key because ABORT error usually occurs in the early phase of multicast communication

  33. Multicast Timeout (ABORT) Error • Typical ABORT error during connection phase in a large environment • Sender tries to complete multicast connection with all receivers within a short period • Network is overloaded and it causes a timeout error • Need to spread the data out during multicast connection by backofftm, connwtout, and connrtry • ABORT error does not occur in a small environment

  34. Multicast Timeout (ABORT) Error • backofftm parameter • Can be set which will tell receivers to pick a random number between 0 ~ backofftm and wait that amount of time before sending a reply • The backofftm spreads the packets out. • This reduces congestion and gives sender time to easily handle the packets from receivers • As a result, it will improve the throughput of the entire distribution • The backofftm should be less than 65535 • When increasing backofftm, the *tout settings (connwtout, dtwtout, and relwtout) should be larger than backofftm. Otherwise timeout will occur • If there are numerous targets, the default backofftm (100 ms) may be too short to handle multicast communication

  35. ADVT_TIMEOUT Error • Wait Timeout (ADVT_TIMEOUT) is receiver side timeout • ADVT_TIMEOUT occurs when receiver reaches timeout after multicast advertisement phase • If multicast data transmission terminated with an error on an endpoint and you do not see ABORT packet in the lcfd.log file, it might be caused by ADVT_TIMEOUT • If you see the following message in the lcfd.log file, it indicates that ADVT_TIMEOUT occurred during multicast communication Feb 04 18:25:13 3 RMTP 97 Wait timeout occurred with [15500] msec. Timeout value [15000]

  36. ADVT_TIMEOUT Error • How to resolve ADVT_TIMEOUT • ADVT_TIMEOUT is defined in mcast_receiver.cfg file on endpoint • By default ADVT_TIMEOUT = 15000 (ms) • ADVT_TIMEOUT can be resolved by increasing ADVT_TIMEOUT in mcast_receiver.cfg file • Try ADVT_TIMEOUT = 20000 or higher and see what happens • If you still see ADVT_TIMEOUT, then increase ADVT_TIMEOUT furter • If you increase connwtout and connrtry, ADVT_TIMEOUT also will need to be increased ADVT_TIMEOUT >= connwtout * connrtry

  37. ADVT_TIMEOUT Error • mcast_receiver.cfg file • Endpoint will look at mcast_receiver.cfg file on gateway anytime multicast receiver is started • If the version of gateway’s mcast_receiver.cfg is different from the endpoint’s, then the endpoint will download the gateway version • mcast_receiver.cfg file is located at $BINDIR/../lcf_bundle.40/bin/<interp>/TMF/LCF • Modifying parameters in mcast_receiver.cfg is not recommended unless you encounter ADVT_TIMEOUT. Normally default parameters should work for your environment

  38. ADVT_TIMEOUT Error • How to modify mcast_receiver.cfg file • If multicast is not enabled on the gateway • Modify the mcast_receiver.cfg on the gateway, then when multicast is enabled the endpoint will download the updated mcast_receiver.cfg • If multicast is already enabled on the gateway • Modify the mcast_receiver.cfg, and the next time the endpoint logins, it will get the updates Or • Modify the mcast_receiver.cfg, then run the following commands. This will restart receivers, which will cause them to download the updated mcast_receiver.cfg • wmdist –s <repeater_name> endpoint_multicast=FALSE • wmdist –s <repeater_name> endpoint_multicast=TRUE • On the Endpoint : Make a copy of mcast_receiver.cfg and name it mcast_receiver.conf for changes to be used on an individual endpoint: $LCF_DATDIR/mcast/mcast_receiver.conf BUT ONLY INCLUDE THOSE VARIABLES THAT ARE TO BE OVERRIDDEN: For example, on an endpoint with multiple NICs, it may be necessary to specify IFADDR. After copying mcast_receiver.cfg to mcast_receiver.conf, remove (or comment out with "#") all other variables except IFADDR.

  39. Execute_timeout Error • Execute_timeout error is one of typical problems in multicast installation phase especially when distributing large data • Execute_timeout occurs only after multicast data transmission is completed • Execute_timeout is used to specify the amount of time that an endpoint method has to return a result after data distribition • When sending a large software package, the software package installation (copy) process may hit execute_timeout • In multicast distribution execute_timeout can occur even if there is no user program defined in the software package • When execute_timeout occurs, the distribution status is changed from WAITING to INTERRUPTED • The following log message can be seen in lcfd.log file 1 21 21:15:42 Q cm_operation_ep2 mdist: Unable to find entry for segment 1gb^1. 1 21 21:15:42 Q MethInit method returned.

  40. Execute_timeout Error • How to resolve execute_timeout error • Execute_timeout can be configured via wmdist command • wmdist –s execute_timeout=time • By default, execute_timeout = 300 (sec) • If you planning to distribute large software packages (greater than 1 GB), increase execute_timeout and ensure that execute_timeout is large enough • Perform distribution test with 2 GB software package (distribution itself cannot be larger than 2GB) • For instance, start with execute_timeout=2000

  41. Time-to-Live (TTL) Count • wmcast command allow you to specify Time-to-Live (TTL) count • wmcast –s <repeater_name> ttl=count • The default value is 5 • When a packet is passed a router, TTL count is decremented; when it reaches 0, the packet is dropped • Need to specify a number larger than the number of routers between sender and receiver (255 is maximum) • In a large-scale environment, the default value is sometimes too small • Check how many hops your network requires prior to distribution (i.e. by ping command) • Some network device, such as VLAN, may deal with TLL count in a different manner

  42. Multicast Performance Tuning • Recommended performance tuning points in multicast data distribution • Use different Advertisement address along with mclow-mchigh range • Multicast bandwidth rate (mc_netload) • Multicast data transmission and MTU • Local disk copy • Multicast buffer • Operating systems, network, etc. • Look for the best configurations and parameters for your environment

  43. MCADVERT, MCHIGH, MCLOW • In large env changing MCADVERT, MCHIGH and MCLOW will improve performance • Provides separate groups for targets below different gateways • Example: Mcadvert: 224.0.1.118  224.10.1.1 Mclow: 224.2.1.18  224.10.1.2 Mchigh 224.2.1.255  224.10.1.255 • Change the above values to different subnets on different gateways • All three addresess can be changed using following commands wmcat –s <repeater_name> mcadvert=value wmcat –s <repeater_name> mchigh=value wmcat –s <repeater_name> mclow=value

  44. Multicast Bandwidth (mc_netload) • Multicast data transmission speed can be configured via mc_netload parameter • wmcat –s <repeater_name> mc_netload=value • By default mc_netload = 500,000 (bytes/sec) • Tuning multicast bandwidth (mc_netload) is good starting point for multicast performance tuning • If mc_netload is set too high, many packets will be dropped and the dropped packets need to be resent which causes performance to suffer • Look for the best bandwidth for your environment • Link speed • Software package size • Background traffic

  45. Multicast Bandwidth (mc_netload) How to determine the best value for mc_netload parameter • Start with the default value (500 kb/s) and measure data distribution performance • Increase and decrease mc_netload, and measure data distribution performance. For instance, try mc_netload=600 (kb/s) and 400 (kb/s) • If you see significant improvement with either mc_netload=400 (kb/s) or 600 (kb/s), repeat step 2 with different bandwidth and see the differences • To find best value for mc_netload, try a couple software packages (data size) for step 1–3 • If you do not see significant difference between step 1 and step 2, the default value is the appropriate value for the environment • A large software package is recommended for mc_netload tuning • mc_netload tuning should be done in production network because background traffic will affect multicast data distribution performance • To make further analysis, capture all distribution data such as mcast.log, wmcast, wmdist command output, etc. • There are network router settings to increase the priority of UDP packets and this setting may affect multicast performance

  46. Multicast Bandwidth (mc_netload) • Analyzing dropped packets • To recognize your network characteristics, analyzing dropped packets is recommended • Leverage the distribution log files captured during mc_netload tuning test • Key information can be seen in multicast log files • Elapsed time of distribution (not including installation) • How many times data was resent • This information should be able to help you in configuring and tuning mc_netload properly

  47. Receive Multicast Local Disk Copy • When distributing a large software package, the installation process will take a long time • Multicast receiver stores the entire data in a local file (depot) , then the data is copied to target directory • To improve installation process performance, secure enough free disk space • Enable multicast to store at least twice the size of distribution data, preferably three times or more • Avoid disk fragmentation on target endpoints • Multicast installation process internally performs a ‘local copy’ from the depot to the installation target directory • Software package is uncompressed during the installation • By default, depot is located at $LCF_DATDIR/depot

  48. Multicast Buffer • Tivoli multicast provides parameters that enable you to configure buffer size of UDP socket on sender and receiver sides • rcvbufsz and sndbufsz parameter on sender • The default value is 250,000 bytes • rcvbufsz and sndbufsz can be configured via wmcast command • wmcast –s <repeater_name> rcvbufsz=size • RCVBUFSZ and SNDBUFSZ parameter on receiver • Defined in mcast_receiver.cfg file and the default value is 98304 bytes • Increasing RCVBUFSZ may result in fewer dropped packets and therefore a reduced distribution time • If you see the following message in receiver side log file, it would be better to increase buffer size. This message indicates that receiver dropped packets because of buffer flow • Multicast buffer is related to setting of UDP socket buffer on operating system • Different operating systems have different limits for the buffer size • Multicast buffer size cannot be larger than operating systems’ limit • Multicast buffer tuning is optional and should be done after multicast data distribution is stabilized Jun 30 15:07:57 3 RMTPERR [RMTPtblWriteUserData_R] buffer size is too small [421]

  49. Multicast Performance Evaluation • When evaluating Tivoli multicast performance, probably the following are the typical scenarios • Before and after performance tuning • Multicast and unicast • Tivoli multicast and the other product’s multicast • Multicast data transfer rate may not seem high. However, assuming there are numerous distribution targets, you should consider how long the same data distribution takes via unicast • As long as you have numerous targets, multicast should be able to show you better throughput than unicast, especially when a large data is distributed

  50. Conclusion • Multicast very much depends on the environment • Start with default values and configuration • Perform multicast tuning and testing in the production environment • Understand the characteristics of the network, distributed data, and targets

More Related