1 / 18

Make Your Code File Driven Methods to let SAS collect file names in your system

Learn how to leverage file-driven approaches in SAS to enhance usability, dynamic output, and flexibility in data processing. Explore SAS 'D-' functions and unnamed SAS pipes for efficient file handling and external data integration.

pgloria
Download Presentation

Make Your Code File Driven Methods to let SAS collect file names in your system

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Make Your Code File Driven Methods to let SAS collect file names in your system Lu Zhang Beijing, China

  2. Introduction Background • SAS Data-Driven Features • Most of the SAS powerful functions are based on a data-driven approach. • Increased general usability as a result of independence from raw data. • Dynamic output from different data contents. • Fewer parameters.

  3. Introduction Background • Idea Needs Illumination Data-Driven feature is self proved to be an excellent approach. To extend it - Let the driven approaches not only based on the data structure but also on the external files. • File driven there it is. • Let SAS communicate with our operating system.

  4. Introduction Background • How This Idea Serves in Pharmaceutical Industry • To import external data. • To combine output files. • Make our macros more flexible. • Example (SDTM implementation with raw data being external *.csv files) • %mimport(ae.csv); • %mimoort(cm.csv); VS %mimport("folder path"); • %mimport(mh.csv); • ... • No need to type their names one by one, just tell SAS where they are.

  5. Introduction Aim of this paper • Two Methods to implement the file driven approach • SAS 'D-' functions. • Using a series of SAS functions including DOPEN, DNUM and DREAD. • Unnamed SAS Pipes • SAS Pipes allows us using DOS commands in Windows system outside SAS.

  6. Method 1: SAS 'D-' Functions Descriptions of Functions • How These 'D-' Functions Works • DOPEN • to open a directory and return a directory identifier value. • DNUM • to get the numbers of members in the opened directory. • DREAD • to return the names of each members. • DCLOSE • to close the opened directory to release the thread.

  7. Method 1: SAS 'D-' Functions Utilizations of Functions Utilization of These Functions and Implementation SAS Code • All should be within a data step. • Open a specific path under which our target files are located with DOPEN. • Get the total number of files exist under the path with DNUM. • In a do-loop, get each files' filename with DREAD and assign them to a variable.

  8. Method 1: SAS 'D-' Functions Results of Method 1 Result Source folder Result data

  9. Method 1: SAS 'D-' Functions Utilization Example (SDTM implementation with raw data being external *.csv files) %macro mimport(path=); **Code for collecting file names; …; %do i=1 %to &k; **Original code for importing files; %end; %mend;

  10. Method 2: Unnamed SAS Pipes Basic Introduction • Basic Introduction of Pipes and Dos Commands • Implementing dos commands within SAS like X statement. • More than X statement, SAS pipes are dynamic connections. • With following SAS code we can look up for dos commands • Example (SAS code to look up for dos commands) • filename indata pipe 'help'; • data help; • infile indata truncover; • input help $300.; • if _n_ ne 1; • run;

  11. Method 2: Unnamed SAS Pipes Utilization of SAS Pipes to Read File Names Implementation SAS Code Example filename indata pipe 'dir "c:\external data\*.csv" /b'; data flst; format fname $30.; infile indata truncover; input fname; call symput("n_file",_n_); run; filename indata clear;

  12. Method 2: Unnamed SAS Pipes Results With Same Case Result By Using SAS Pipes Exactly same, but less heavy code and simpler logic compared with using 'D-' functions.

  13. Method 2: Unnamed SAS Pipes Further Extended • UNC Path • UNC - Universal Naming Convention. • Many companies have their global SAS working environment in which network shares often being used. • UNC path is necessary to connect to network share. • It could be the chance that 'dir' may failed to find UNC path.

  14. Method 2: Unnamed SAS Pipes Pipes Under UNC Path • How to deal with the situation that dir failed? • Dos command "subst" - Associates a path with a driver letter. • Steps to establish a virtual driver that dir can find. • Set SAS options 'noxwait'. • Using X statement and the dos command "subst" to assign the target path to a virtual driver. • With virtual driver set up, pipes could work again.

  15. Method 2: Unnamed SAS Pipes Pipes Under UNC Path Example (assign a UNC path "\\global\project\data" to virtual driver) option noxwait; X 'subst n: \\global\project\data'; Example (results)

  16. Discussion & Conclusion Applying in More Fields Example: To compare the latest 2 batches of extracted data. filename indata pipe 'dir “C:\extracted" /b'; data fnam; infile indata truncover; input fnam $200.; fnam2=lag(fnam); call symput('last', strip(fnam2)); call symput('current', strip(fnam)); run; File driven approaches can be more flexible.

  17. Discussion & Conclusion • Now we could: • Importing external data. (CSV XLS ...) • Combining outputs. (RTF LST ... ) • Coordinating with other techniques to function more. • With methods of: • SAS 'D-' functions. • SAS pipes. • In addition, with SAS pipes, we gain • More efficiency. • Less heavy code. • More flexibility.

  18. Q&AThanks

More Related