1 / 22

voldetails

voldetails. An OpenAFS utility, shamelessly derived (mostly copied) from existing source, and integrated in a useful way. History. ITAR: Don’t let the bad guys get your good stuff How do we know who’s seeing what? Let’s run fileserver audit logging and get a sample of what’s going on …

laurel
Download Presentation

voldetails

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. voldetails An OpenAFS utility, shamelessly derived (mostly copied) from existing source, and integrated in a useful way Kim Kimball [dhk@ccre.com]

  2. History • ITAR: Don’t let the bad guys get your good stuff • How do we know who’s seeing what? • Let’s run fileserver audit logging and get a sample of what’s going on … • 70-90% of operations: anonymous, so let’s • Freak out • Calm down: almost all are from within our networks • Worry: are we giving away the good stuff • Brain storm/f*rt: Let’s record all IO operations! • What’s a FID? Kim Kimball [dhk@ccre.com]

  3. fileserver -audit • Running fileserver with –audit <auditlog> records IO operations in a log file or FIFO • Runs continuously, provides continuous trace • Who did it • When they did it • What they did to it (create/stat/read/write/delete) • Which file/directory/ACL they did it to • Why they did it (NOT!) • From which client system Kim Kimball [dhk@ccre.com]

  4. Logged operations src/audit/audit.h • #define FetchDataEvent "AFS_SRX_FchData" • #define FetchACLEvent "AFS_SRX_FchACL" • #define BulkFetchStatusEvent "AFS_SRX_BFchSta" • #define FetchStatusEvent "AFS_SRX_FchStat" • #define StoreDataEvent “AFS_SRX_StData" • #define StoreACLEvent "AFS_SRX_StACL" • #define StoreStatusEvent "AFS_SRX_StStat" • #define RemoveFileEvent “AFS_SRX_RmFile" • #define CreateFileEvent “AFS_SRX_CrFile" • #define RenameFileEvent "AFS_SRX_RNmFile" • #define SymlinkEvent "AFS_SRX_SymLink" • #define LinkEvent "AFS_SRX_Link" • #define MakeDirEvent "AFS_SRX_MakeDir" • #define RemoveDirEvent "AFS_SRX_RmDir" • #define SetLockEvent "AFS_SRX_SetLock" • #define ExtendLockEvent "AFS_SRX_ExtLock" • #define ReleaseLockEvent "AFS_SRX_RelLock" • #define GetVolumeStatusEvent "AFS_SRX_GetVolS" • #define SetVolumeStatusEvent "AFS_SRX_SetVolS" • #define FlushCPSEvent "AFS_SRX_FlusCPS" Kim Kimball [dhk@ccre.com]

  5. Examples: audit output • When … What Op …. Code/result … username/cell… client IP … userID … FID (volume ID:vnode:uniquifier)** • Fri May 21 09:18:24 2010[48] EVENT AFS_SRX_StData CODE 0 NAME bstark@CCRE.COM HOST 70.57.68.116 ID 40714 FID 3249855686:130:2604 ** Kim Kimball [dhk@ccre.com]

  6. We’d generally not like to see … • Sat May 22 19:08:03 2010 [12] EVENT AFS_SRX_StData CODE 0 NAME --UnAuth-- HOST 70.57.60.57 ID 32766FID 1659493657:5994:230200 Kim Kimball [dhk@ccre.com]

  7. … But if we do Is it part of the good stuff? (We’d really like to know what and where FID 1659493657:5994:230200 is) Kim Kimball [dhk@ccre.com]

  8. How to … • Obviously OpenAFS reads all this info from the vice partitions/volumes • We can see file names, directories, ACLs, get volume metadata … • So we’ll find the pieces we want and glue them together, fail to compile/link, dump core, eat kale, baste a turkey Kim Kimball [dhk@ccre.com]

  9. What we’re doing • Open a volume header from disk • Map to volume structure • Open and scan small vnode index • Files • Mount points (sym link whose name starts with #) • Open and scan large vnode index • Directories • ACLS Kim Kimball [dhk@ccre.com]

  10. What we’re doing (more detail) • /vicepX/<volid> passed in from command line • Get volume ID from string, atoi() to convert to numeric • get "volparse id", a numeric partition ID, an ordinal representation of vicep where vicepa is 0, vicepb is 1, ... * • Get vicep from string • Check volume header file exists, or die • Open volume header file, or die • Read volume header file into volumeDiskHeaderstruct, or die • Read volume header file from disk. • GetVolumeInfo() populates large/small Vnode Indexes • Scan large vnode indexes to get directory entries and ACLs, • Scan small vnode indexes to get file entries and mount points (which are sym links whose link-from name starts with # Kim Kimball [dhk@ccre.com]

  11. What we get … • voldetails • Quickly provides useful information about a given volume • Directories in the volume by name and FID • Paths relative to root node of volume • Files in the volume by name and FID • Mount points • Access Control Lists (ACLs) at time of enumeration • And we can use it to glue the FIDs in the audit log to paths/names Kim Kimball [dhk@ccre.com]

  12. Sources used • static char *rights2str(afs_uint32 rights) /* From src/tests/parsevnode.c */ • scanLargeVnode(dev, node, partitionName, option) /* FROM: src/vol/test/listVicepx.c */ • createDirEnt(dirEntry, fileName, vnode, unique) /* FROM: src/vol/test/listVicepx.c */ Kim Kimball [dhk@ccre.com]

  13. Sources used • printDirs(partitionName) /* FROM: src/vol/test/listVicepx.c */ • scanSmallVnode(dev, node, partitionName, option) /* FROM: src/vol/test/listVicepx.c */ • GetVolumeInfo(int device, struct VolumeHeader * vheader) /* FROM: src/volser/vos.c*/ Kim Kimball [dhk@ccre.com]

  14. Voldetails is fast • Operates directly on /vicepX/xyz.vol • Does not present a load to the fileserver or volserver: no visible impact on performance • Avoids recursion due to mount points embedded in volume Kim Kimball [dhk@ccre.com]

  15. A run … # timex ./voldetails -n /vicepa/V0536878650.vol > /var/tmp/voldetails.out real 3.07 user 2.59 sys 0.27 # # grep ^directory= /var/tmp/voldetails.out | wc -l 7147 # grep ^file= /var/tmp/voldetails.out | wc -l 172026 # Kim Kimball [dhk@ccre.com]

  16. voldetailsoutput: directories volume=536878650,volumename=user.k.kim directory=/,fid=536878650:1:1,positiveacl=-204 rlidwka,positiveacl=-101 rlidwk,positiveacl=10056 rlidwka,positiveacl=12761 rlidwka directory=/.solregis,fid=536878650:1323:81822,positiveacl=-204 rlidwka,positiveacl=-101 rl,positiveacl=12761 rlidwka,positiveacl=100023 rlidwka directory=/ONLYKIM,fid=536878650:8539:1152301,positiveacl=12761 rlidwka directory=/EXPORTFS,fid=536878650:489:236764,positiveacl=-204 rlidwka,positiveacl=-101 liw,positiveacl=12761 rlidwka directory=/1.4.11,fid=536878650:8577:1152305,positiveacl=-204 rlidwka,positiveacl=-101 rlidwk,positiveacl=10056 rlidwka,positiveacl=12761 rlidwka directory=/ACLTEST3,fid=536878650:7537:1103076,positiveacl=-204 rlidwka,positiveacl=-101 rlidwka,positiveacl=12761 rlidwka Kim Kimball [dhk@ccre.com]

  17. voldetailsoutput: files file=/smb.conf,fid=536878650:346566:1278042 file=/afsfsa,fid=536878650:346568:1278139 file=/afsfs02b,fid=536878650:346570:1278140 file=/foo,fid=536878650:346572:1278145 file=/voldetails,fid=536878650:346578:1278148 Kim Kimball [dhk@ccre.com]

  18. voldetailsoutput: mount points mountpoint=#test.afs.eraseme.,volume=536878650,directory=/MYTESTAFS/testafs,fid=536878650:239672:1149912 mountpoint=#root.cell.,volume=536878650,directory=/adir/adir2/myrootcell,fid=536878650:239814:1152298 Kim Kimball [dhk@ccre.com]

  19. Uses of voldetails+ audit log • Track changes to ACLs if, say, there are restrictions on anonymous access (system:anyuser) • Track deletions/modifications: who did it and when from which client • Forensics • Either gently inform or spank errant users • How much of your activity is by anonymous users, inside/outside your networks • Which can be scary Kim Kimball [dhk@ccre.com]

  20. Audit logging on lots of operations • Millions of operations per week • Lots of audit output • Restrict output to operations of interest by modifying fileserver source Kim Kimball [dhk@ccre.com]

  21. Examples: volinfooutput • VOLINFO Gives FIDS, and UFS file names, and is fast • # timexvolinfo -filenames -part /vicepa -volumeid 536878650 > volinfo.out • real 6.74 • user 4.61 • sys 1.24 • 9216 Vnode 73.25587.1 cloned: 1, length: 2048 linkCount: 2 parent: 103 UFS-Filename: /vicepa/AFSIDat/u/us=+U/+/+/7=++2AzM • 9472 Vnode 75.37230.991 cloned: 1, length: 2048 linkCount: 11 parent: 3 UFS-Filename: /vicepa/AFSIDat/u/us=+U/+/+/9=++2sKY • 9728 Vnode 77.37640.73 cloned: 1, length: 2048 linkCount: 3 parent: 75 UFS-Filename: /vicepa/AFSIDat/u/us=+U/+/+/B=++2UkY • 9984 Vnode 79.37861.4 cloned: 1, length: 2048 linkCount: 2 parent: 75 UFS-Filename: /vicepa/AFSIDat/u/us=+U/+/+/D=++2IyY • # • CHANGE AUDIT EVENTS • src/viced/afsfileprocs.c • voldetails: Orphaned files, salvage cleared • file=/ORPHANED_NoUnique,fid=536878650:133114:870398 • file=/ORPHANED_NoUnique,fid=536878650:133116:870399 • file=/ORPHANED_NoUnique,fid=536878650:133118:870400 • file=/ORPHANED_NoUnique,fid=536878650:133120:870401 • file=/ORPHANED_NoUnique,fid=536878650:133122:870402 Kim Kimball [dhk@ccre.com]

  22. find_mps • FIND_MPS • # timexfind_mps • ./root.afs => #root.afs • ./root.cell => #root.cell • Can't cd to (./MISC/) test : Permission denied • ./MISC/itecduplicate => #itecduplicate • Can't cd to (./MISC/) foo : Permission denied • Can't cd to (./MISC/) WWWtest : Permission denied • ./MISC/VOLSCAN/hayes => #user.d.dog • ./MISC/TEST/ccre02 => #ccre02 • ./MISC/SyncVolStructs/src.4 => #user.mary.backup • ./MISC/SyncVolStructs/src.5 => #user.dog.backup • ./MISC/SyncVolStructs/target.5 => #user.dog • ./MISC/SyncVolStructs/target.4 => #user.mary • ./MISC/SyncVolStructs/x => #root.cell • ./MISC/SyncVolStructs/src.6 => #user.horton.backup • ./MISC/SyncVolStructs/target.6 => #user.horton • ./MISC/SyncVolStructs/target.3 => #user.hismajesty • Can't cd to (./MISC/SyncVolStructs/) secure : Permission denied • ./MISC/SyncPTS/gp.afs => #gp.afs • Can't cd to (./MISC/ITEC/GENS/SCRIPTS/SETUP/) FOOBAR : Permission denied • ./MISC/BACKUPS/NEWVOLSETS/user => #user • ./MISC/BACKUPS/TESTCPU03/TESTccre=> #TESTccre • ./KRB03/testafs => #test.afs.eraseme • Can't cd to (./) X : Permission denied • ./public/kimprod => #ccre.com:user.me • real 32.54 • user 2.12 • sys 11.04 Kim Kimball [dhk@ccre.com]

More Related