1 / 28

Sep. 21-22, 2006

v. FME Worldwide User Conference - Vancouver. Sep. 21-22, 2006. Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen Spatial, Australia. Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith

porter-cobb
Download Presentation

Sep. 21-22, 2006

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. v FME Worldwide User Conference - Vancouver Sep. 21-22, 2006 Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith, Lagen Spatial, Australia

  2. Using FME Objects Python API to Provide Platform Independent Translations Andrew Smith Spatial Technical Consultant Lagen Spatial - Australia

  3. Using FME Objects Python API to Provide Platform Independent Translations Introduction • Who am I? • Project Overview Python and FMEObjects • Method for Development • Pitfalls • Tips and Tricks Output Examples Questions

  4. Using FME Objects Python API to Provide Platform Independent Translations Who am I? Name: Andrew Smith Company: Lagen Spatial Pty Ltd Official Title: Spatial Technical Consultant Email: andrew@lagenspatial.com.au

  5. Using FME Objects Python API to Provide Platform Independent Translations Project Overview • Geoscience Australia • plays a critical role by producing first-class geoscientific information and knowledge. This can enable the government and the community to make informed decisions about the exploration of resources, the management of the environment, the safety of critical infrastructure and the resultant wellbeing of all Australians. www.ga.gov.au

  6. Using FME Objects Python API to Provide Platform Independent Translations Project Overview • Geographic Data Management Division • Solution Name - Geoflex • Project Leader - Tony Hunter • Formats Supported • ESRI Shape File (.shp, .shx, .dxf) • Personal Geodatabase • Mapinfo Native Format (.tab), Mapinfo Data Interchange Format (.mif), and • Geographic Mark-up Language (.gml)

  7. Using FME Objects Python API to Provide Platform Independent Translations Project Overview • Requirements • Be platform independent i.e. the core extraction process is able to execute on both Windows and Unix Operating Systems • Allow a User to Select the Feature Dataset to be extracted • Allow a User to Extract the datasets to on 3 scales • National • State/Province • Map Tile • Custom Symbology Developed by GA to be applied to the Output Datasets • Dynamically update the list of Feature Datasets that are available for extraction through interrogating the underlining Spatial DBMS schema

  8. Using FME Objects Python API to Provide Platform Independent Translations Platform Independent Translations with FMEObject for Python • Method • FME Workbench to Develop the Translation Process

  9. Using FME Objects Python API to Provide Platform Independent Translations

  10. Using FME Objects Python API to Provide Platform Independent Translations • Copy the Transformers into a text editor to get the FME Script that defines Translation

  11. ArcSDEQuerier MACRO wheremac SDE_SEARCH_METHOD_FIELD __SEARCH_METHOD INCLUDE [ if { {SDE_AREA_INTERSECT} == {SDE_NONE} } { puts {MACRO wheremac}} ] FACTORY_DEF * TeeFactory \ FACTORY_NAME ARCSDEQUERIER_Globber \ INPUT FEATURE_TYPE clip_area_0 \ OUTPUT FEATURE_TYPE __GLOBBED__ FACTORY_DEF * SDE30QueryFactory \ FACTORY_NAME ARCSDEQUERIER_Querier \ INPUT FEATURE_TYPE __GLOBBED__ \ @SupplyAttributes(__SERVER,lagen-pc-1,__INSTANCE,esri_sde,__DATASET,sde,__USERID,sde,__PASSWORD,sdeuser,__TARGET_TABLE,STONNING_CAD,__SEARCH_METHOD,SDE_AREA_INTERSECT,__WHERE_CLAUSE,"",__VERSION,SDE.DEFAULT) \ SDE_SERVER_FIELD __SERVER \ SDE_INSTANCE_FIELD __INSTANCE \ SDE_DATASET_FIELD __DATASET \ SDE_USERID_FIELD __USERID \ SDE_PASSWORD_FIELD __PASSWORD \ SDE_VERSION_NAME_FIELD __VERSION \ SDE_TARGET_TABLE_FIELD __TARGET_TABLE \ $(wheremac) \ WHERE_CLAUSE_FIELD __WHERE_CLAUSE \ QUERY_MODE QUERY \ SEARCH_ORDER SPATIAL_FIRST \ COMBINE_ATTRIBUTES ADD \ OUTPUT_DUPLICATES NO \ GET_SPATIAL_RELATIONS \ OUTPUT RESULT FEATURE_TYPE * \ @RemoveAttributes(__SERVER,__INSTANCE,__DATASET,__USERID,__PASSWORD,__TARGET_TABLE,__SEARCH_METHOD,__WHERE_CLAUSE,__VERSION) \ @SupplyAttributes(_table_name,@FeatureType()) \ @FeatureType(@Concatenate(ARCSDEQUERIER,"_",@FeatureType())) \ @Transform(SDE30,FME_GENERIC)

  12. ClippingFactory FACTORY_DEF * ClippingFactory \ FACTORY_NAME CLIPPER \ INPUT CLIPPER FEATURE_TYPE clip_area_1 \ INPUT CLIPPEE FEATURE_TYPE ARCSDEQUERIER_STONNING_CAD \ MULTICLIP YES \ CLIPPEE_ON_BOUNDARY INSIDE \ MERGE_CLIPPER_ATTRIBUTES No \ DO_NOT_AGGREGATE NO \ CLIPPER_ATTR_PREFIX "clipper_" \ OUTPUT INSIDE FEATURE_TYPE CLIPPER_INSIDE \ OUTPUT CLIPPED_INSIDE FEATURE_TYPE CLIPPER_CLIPPED_INSIDE \ OUTPUT EXTRA_CLIPPER FEATURE_TYPE CLIPPER__delete__ @Tcl2("FME_LogMessage fme_warn {Deleting extra clipper feature encountered which entered transformer CLIPPER}") \ OUTPUT NONPOLY_CLIPPER FEATURE_TYPE CLIPPER__delete__ @Tcl2("FME_LogMessage fme_warn {Deleting non-polygonal clipper feature which entered transformer CLIPPER}") FACTORY_DEF * TeeFactory \ FACTORY_NAME CLIPPER_Nuker \ INPUT FEATURE_TYPE CLIPPER__delete__ \ @Log("Offending feature was:")

  13. Joiner (Relate) ‘Factory’ Relate CACHE_SIZE "JOINER:all_symbology_lookup_table" 5000 Relate TABLE_LOCATION "JOINER:all_symbology_lookup_table" "d:\User Documents\LagenSpatial\Projects\GA - FME Data Migration\Symbology Library (new)\version1.1 20060419\all_symbology_lookup_table.csv" Relate TABLE_DEF "JOINER:all_symbology_lookup_table" CSV \ FACTORY_DEF * TeeFactory \ FACTORY_NAME JOINER \ INPUT FEATURE_TYPE CLIPPER_INSIDE \ INPUT FEATURE_TYPE CLIPPER_CLIPPED_INSIDE \ OUTPUT FEATURE_TYPE JOINER_JOINED \ @Relate(JOINER,Read) \ @RenameAttributes(_matched_records,matched_records)

  14. Using FME Objects Python API to Provide Platform Independent Translations • Save the pasted output as one or more .fmi files • Use the FMEFactoryPipeline.addFactoriesFromFile() method to include your translation pipeline .fmi files into your PyFMEObjects Script.

  15. Using FME Objects Python API to Provide Platform Independent Translations • WHY??? • Reduce possible errors in your script • Special Characters • \n – New line • \r – Carriage Return • \w – White Space • \t – Tab !!! • Special characters MEAN something in both Python and FME Mapping Files

  16. Using FME Objects Python API to Provide Platform Independent Translations • Allows you to solve other bugs • Logical • Programmatic • Data Flow • Logical – • If…Then…Else • Do…Loop • Switch…Case

  17. Using FME Objects Python API to Provide Platform Independent Translations • Allows you to solve other bugs • Logical • Programmatic • Data Flow • Programmatic – • Incorrect Method use • String Manipulation • Reading Multiple Source Datasets

  18. Using FME Objects Python API to Provide Platform Independent Translations • Allows you to solve other bugs • Logical • Programmatic • Data Flow • Data Flow – • Combination of Logical and Programmatic Errors • Unexpected output

  19. Using FME Objects Python API to Provide Platform Independent Translations • Once you have your Python script reproducing the same output as Workbench, its time to Embed your Factories • …which is also where the fun begins!!

  20. Using FME Objects Python API to Provide Platform Independent Translations • First of all… • Use the Python Lists i.e. • stringListVariable = [‘green’, ‘eggs’, ‘and’, ‘ham’] • Cuts out the possibility of naughty special characters sneaking their way into your Factory Definition • i.e. DON’T USE THE “””green eggs and ham“”” strings – it tells Python to use the string as it is, Naughty Special Characters and all

  21. GOOD fme_session.config(['Relate', 'RELATION_DEF', 'JOINER', '1:0..1+', 'TABLE', 'JOINER:all_symbology_lookup_table', 'UNIQUE(Esri_No,feature_type)', 'JOIN', 'Esri_No', 'TO', 'SYMBOL', 'JOIN', 'feature_type', 'TO', 'fme_type', 'MATCHED_RECORDS_ATTR', 'matched_records', 'TRIM_TRAILING', 'No', 'TRANSFER', 'mapinfo_brush_background', 'TO', 'mapinfo_brush_background', 'TRANSFER', 'mapinfo_brush_foreground', 'TO', 'mapinfo_brush_foreground', 'TRANSFER', 'mapinfo_brush_pattern', 'TO', 'mapinfo_brush_pattern', 'TRANSFER', 'mapinfo_pen_color', 'TO', 'mapinfo_pen_color', 'TRANSFER', 'mapinfo_pen_pattern', 'TO', 'mapinfo_pen_pattern', 'TRANSFER', 'mapinfo_pen_width', 'TO', 'mapinfo_pen_width', 'TRANSFER', 'mapinfo_symbol_angle', 'TO', 'mapinfo_symbol_angle', 'TRANSFER', 'mapinfo_symbol_color', 'TO', 'mapinfo_symbol_color', 'TRANSFER', 'mapinfo_symbol_file_name', 'TO', 'mapinfo_symbol_file_name', 'TRANSFER', 'mapinfo_symbol_font', 'TO', 'mapinfo_symbol_font', 'TRANSFER', 'mapinfo_symbol_shape', 'TO', 'mapinfo_symbol_shape', 'TRANSFER', 'mapinfo_symbol_size', 'TO', 'mapinfo_symbol_size', 'TRANSFER', 'mif_brush_background', 'TO', 'mif_brush_background', 'TRANSFER', 'mif_brush_foreground', 'TO', 'mif_brush_foreground', 'TRANSFER', 'mif_brush_pattern', 'TO', 'mif_brush_pattern', 'TRANSFER', 'mif_pen_color', 'TO', 'mif_pen_color', 'TRANSFER', 'mif_pen_pattern', 'TO', 'mif_pen_pattern', 'TRANSFER', 'mif_pen_width', 'TO', 'mif_pen_width', 'TRANSFER', 'mif_symbol_angle', 'TO', 'mif_symbol_angle', 'TRANSFER', 'mif_symbol_color', 'TO', 'mif_symbol_color', 'TRANSFER', 'mif_symbol_file_name', 'TO', 'mif_symbol_file_name', 'TRANSFER', 'mif_symbol_font', 'TO', 'mif_symbol_font', 'TRANSFER', 'mif_symbol_shape', 'TO', 'mif_symbol_shape', 'TRANSFER', 'mif_symbol_size', 'TO', 'mif_symbol_size',])

  22. BAD fme_pipeline.addFactoryFromString( """FACTORY_DEF * SDE30QueryFactory \ FACTORY_NAME ARCSDEQUERIER_Querier \ INPUT FEATURE_TYPE __GLOBBED__ \ @SupplyAttributes(__SERVER,""" + server + """,__INSTANCE,port:""" + instance +""",__DATASET,""" + _dataset + """,__USERID,""" + username + """,__PASSWORD,""" + password + """,__TARGET_TABLE,""" + _owner + "." + featuredataset.upper() + """,__SEARCH_METHOD,SDE_ENVELOPE,__VERSION,SDE.DEFAULT) \ SDE_SERVER_FIELD __SERVER \ SDE_INSTANCE_FIELD __INSTANCE \ SDE_DATASET_FIELD __DATASET \ SDE_USERID_FIELD __USERID \ SDE_PASSWORD_FIELD __PASSWORD \ SDE_VERSION_NAME_FIELD __VERSION \ SDE_TARGET_TABLE_FIELD __TARGET_TABLE \ SDE_SEARCH_METHOD_FIELD __SEARCH_METHOD QUERY_MODE QUERY \ SEARCH_ORDER OPTIMIZE \ COMBINE_ATTRIBUTES MERGE \ GET_SPATIAL_RELATIONS \ OUTPUT RESULT FEATURE_TYPE ARCSDEQUERIER_OUTPUT \ @RemoveAttributes(__SERVER,__INSTANCE,__DATASET,__USERID,__PASSWORD,__TARGET_TABLE,__SEARCH_METHOD,__WHERE_CLAUSE,__VERSION) \ @SupplyAttributes(_table_name,@FeatureType()) \ @Transform(SDE30,FME_GENERIC)""" )

  23. Using FME Objects Python API to Provide Platform Independent Translations • Why Use Python? • Great ‘Whipitupability’ • It is an ‘interpreted’ language • Its free and powerful!!! With a great open source community • Supported by ESRI and FME • You can write your own transformers for FME

  24. Using FME Objects Python API to Provide Platform Independent Translations Example - Sydney

  25. Using FME Objects Python API to Provide Platform Independent Translations Example - Darwin

  26. Using FME Objects Python API to Provide Platform Independent Translations Example - Perth

  27. Using FME Objects Python API to Provide Platform Independent Translations Example – Pt. Lincon

  28. Using FME Objects Python API to Provide Platform Independent Translations QUESTIONS??? Name: Andrew Smith Email: andrew@lagenspatial.com.au Phone: +61 2 9545 7740

More Related