David York Software Design Engineer Printing and Imaging davidyor@microsoft.com

1. WIA Enhancements And Driver Development David York Software Design Engineer Printing and Imaging davidyor@microsoft.com All sample code subject to change

2. Contents • New Longhorn Scanner Features - Filters • Segmentation Filter • Image Processing Filter • Longhorn Scanner Item Tree • Windows XP WIA item tree • Item tree structure • Item tree example • Generated application items • Compatibility Layer • Longhorn WIA Drivers – Implications • New Data Transfer Model • Stream-based transfers • Multi-page transfers • Sample stream-based transfer source code • Windows NT 4.0 Authority\LocalService Security Context • Cleanly Supporting Cancel Operations • Correctly Writing and Validating Properties

3. New Longhorn Scanner Features – Filters

4. New Longhorn Scanner Features – Filters • Longhorn will support two types of WIA scanner filters • Segmentation Filter • Image Processing Filter • Image processing filter is used with new stream-based WIA Transfer Model • Segmentation filter enables hardware vendors to provide region detection code and expose this functionality to WIA application through new interfaces • Image processing code can now be separated from driver code – enables accurate live previews

5. Segmentation Filter • Detects regions in preview image • “Cuts out” the regions creating multiple images • Scenario: Scanning multiple separate photos at the same time from a flatbed scanner

6. Segmentation Filter Application Segmentation Filter Item Tree Segmentation Filter createschild items and set the coordinatesof the sub-images into each item. It does not acquire the images. WIA service Scanner Driver

7. Segmentation Filter • New COM interface IWiaSegmentationFilter: interface IWiaSegmentationFilter : IUnknown { HRESULT DetectRegions( [in] IStream *pInputStream, [in] IWiaItem2 *pWiaItem2); } • pInputStream – contains (preview) image • pWiaItem2 – the WIA item for which the image (pInputStream) was acquired • pInputStream and pWiaItem2 must “match”! • Calls IWiaItem2::GetExtension to create an instance of the segmentation filterpWiaItem2->GetExtension(bstrSegmentationFilter, IID_IWiaSegmentationFilter, (void**) &pFilter); • Calls the filter’s DetectRegions function – typically with a preview stream:pFilter->DetectRegions(pPreviewStrean, pWiaItem2);

8. Segmentation Filter • In-proc COM component (runs in the application’s process) • Microsoft will provide a replaceable segmentation filter for drivers • Segmentation filter must support all image formats that the driver supports! • Simple modification to driver’s INF file[MyDriver.AddReg]…HKCR,CLSID\{UiClassId}\shellex\ SegmentationFilter\{FilterClassId}

9. Image Processing Filter • Used for two purposes • Filtering scanned image to do image processing that can’t be done in hardware • Provide live preview by modifying preview image with realistic estimate of the final scan (no more flowers!) • The live preview is used by the Preview Component, which is written by Microsoft, to perform real live previews • Invoked in the application process by WIA components

10. Image Processing Filter Application Scanner Driver IWiaPreview IWiaTransfer WIA service ImageProcessingFilter

11. Image Processing Filter • Same interface used in both preview and final scan • Needs to implement two interfaces • IWiaImageFilter • IWiaTransferCallback • New COM interface IWiaImageFilter: interface IWiaImageFilter : IUnknown { HRESULT InitializeFilter( [in] IWiaItem2 *pWiaItem2, [in] IWiaTransferCallback *pCb); } • pWiaItem2 – item for which we want to retrieve image • pCb – the application’s callback function

12. Image Processing Filter • In-proc COM component implementing IWiaImageFilter and IWiaTransferCallback (runs in the application’s process) • Needs to support any image format that driver supports • Filter and driver need to read WIA_DPS_PREVIEW to determine what the current purpose is, i.e., preview scan or final scan • Recommended to move filtering functionality from driver to filter • At a minimum all filters should implement brightness and contrast • Simple modification to driver’s INF file[MyDriver.AddReg]…HKCR,CLSID\{UiClassId}\shellex\ ImageProcessingFilter\{FilterClassId}

13. Longhorn Scanner Item Tree

14. Windows XP WIA Item Tree • Applications see the device as a hierarchical tree with a root item and child items • The root item represents the device • The child items represent programmable data sources, data, or folders that contain other items • WIA Minidriver creates Driver Item Tree • Tree composed of driver items (IWiaDrvItem) • WIA Service produces a copy of this tree • Composed of Application items (IWiaItem) • The item tree in Windows XP had only one sub item under root • This is functional for Flatbed scanners • But, not representative of scanners with rich features • A richer item tree needed to represent fuller functionality, e.g., • Multi-region scanning • Film scanning • Storage capabilities

15. Item Tree Structure • In Longhorn, a WIA device is broken down into items by functionality • How do we know which items represent what functionality on the device? • Items belong to a category • Categories are expressed by the property WIA_IPA_ITEM_CATEGORY • Category is used to classify content or supported behavior of a WIA Item • Valid categories are • WIA_CATEGORY_FINISHED_FILE • WIA_CATEGORY_FLATBED • WIA_CATEGORY_FEEDER • WIA_CATEGORY_FILM

16. Item Tree Structure • Just like WIA_IPA_FORMAT, drivers can have custom categories • If you wish to support a data transfer on an item • Mark with WiaItemTypeTransfer flag • Items of the categories listed in the previous slide must be transfer-capable • Must have transfer related WIA properties: tymed, format, … • By specifying a category, you are committing to support a minimal set of properties for that feature • For example, WIA_CATEGORY_FLATBED would include • Properties to specify selection area on the scan bed • Properties for bit depth, color type, format of the resulting scanned image • Etc.

17. Item Tree Example WIA representation Root Flatbed Feeder Film • The following scanner has an automatic document feeder attachment, a film scanner attachment, and a flatbed

18. Item Tree Example WIA representation Root Flatbed Feeder Film • Here is an example of the properties which need to be supported for a specific item; This example shows the flatbed item WIA_CATEGORY_FLATBED WiaItemTypeTransfer WiaItemTypeImage Etc.

19. Generated Application Items • In certain scenarios, app items could be created which are not in the driver item tree • For example, a possible film scanning scenario • User initiates preview of the film strip • Software runs on the preview image to extract individual frames • Final scan is done on each of the frame regions detected • In WIA terms, the scenario is • Calling app initiates preview scan • App runs segmentation filter to detect regions • Each region is represented as a child of the parent item • Child items created by call to CreateChildItem()

20. Generated Application Items • The below example shows the generation of items under a film attachment item. Root Root Root Root Root Film Film Film Film Film Frame 1 Frame 1 Frame 2 Frame 2 Frame 3 Frame 3

21. Generated Application Items Driver Item Tree Driver Item Tree Root Root Film Film Create App Items AppItem Item Tree AppItem Item Tree Root Root Film Film Frame 1 Frame 2 Frame 3

22. Generated Application Items • If you support this functionality, driver should be able to initialize these generated items • These items are marked by WIA with WiaItemTypeGenerated • General rule: Initialize the item the same way as you would the parent • E.g., A child created off of a Flatbed item should be initialized with Flatbed properties • A property is used to indicate whether you support this or not • This functionality also used for multi-region scanning scenarios • Segmentation filter calls CreateChildItem() • To WIA, this is just another application component

23. Compatibility Layer

24. Compatibility Layer • WIA will provide a new COM interface: IWiaItem2 • New items only accessible through IWiaItem2 • New supporting interfaces IWiaDevMgr2 and IEnumWiaItem2 added • IWiaItem2 is a superset of IWiaItem • IWiaItem interface unchanged to guarantee compatibility • IWiaItem2 supports multi-region scanning, IWiaItem does not • IWiaItem2 and IWiaItem support the preview filter • Compatibility layers to ensure that • Windows XP applications work with Longhorn drivers • Longhorn applications work with Windows XP drivers

25. Compatibility Layer Windows XP App Longhorn App IWiaItem IWiaItem2 WIA Windows XP to Longhorn Compat Layer Longhorn to Windows XP Compat Layer Windows XP Driver Longhorn Driver

26. Compatibility Layer • Legacy (IWiaItem applications) will work with Longhorn drivers • Longhorn (IWiaItem2 applications) will work with legacy drivers, but without new Longhorn Functionality • IWiaItem and IWiaItem2 will be automatically created by the WIA Service – no duplicate driver development code is necessary • New drivers only have to support the new transfer type (discussed later) • IHVs: Write drivers exposing the new item tree structure • ISVs: Write applications using the new item interfaces which utilize the enhanced item tree structure

27. Longhorn WIA Drivers – Implications • New functionality enables • Simplified driver development in many areas e.g., data transfers • Looser coupling between value-add components • Example: Driver can be inbox, but enhanced functionality can be achieved through vendor add-ins like Image Processing and Segmentation filters • Enhanced WIA functionality requires • New data transfer model, new WIA item tree structure and new extensions (e.g., filter support) • Therefore, separate drivers recommended for “legacy” WIA (Windows XP) and enhanced WIA (Windows Longhorn) • Is it possible to write one driver for all Operating Systems? • Yes, but legacy Operating Systems will not expose enhanced functionality • Easier to have separate drivers and install different package depending on OS

28. Longhorn WIA Drivers – Implications • New functionality seems to introduce incompatibility with old applications • WIA service will implement a compatibility layer ensuring legacy applications continue to function, even when talking to new drivers • Same goes for new apps talking to legacy drivers • But, clearly, enhanced features not available when using legacy drivers • Any legacy driver which works under new security context will be supported • Through the internal compatibility layer • However, for Longhorn logo, drivers will be required to support: • The new item tree structure • New WIA data transfer; Legacy transfer will not be required • Segmentation filter if required • Example: For software frame detection

29. New Data Transfer Model

30. Stream-Based Transfers • Longhorn WIA drivers will support a new stream-based transfer model • A new interface, IWiaTransfer, is provided for stream-based transfers • Supporting legacy transfers is not required to Longhorn logo a driver • Legacy applications will still work via the application compatibility layer • Drivers implement the body of the new transfer in drvAcquireItemData: • Check the lFlags parameter for WIA_MINIDRV_TRANSFER_DOWNLOAD. If true, the transfer is stream-based • Call QueryInterface on the minidriver callback, requesting IID_IWiaMiniDrvTransferCallback • Call IWiaTransfer::GetNextStream to get an IStream interface, and write data directly to that interface via IStream::Write. • Send progress messages, such as WIA_TRANSFER_MSG_END_OF_STREAM, via IWiaTransfer.

31. Multi-Page Transfers • Multi-page transfers are very cleanly supported with IWiaTransfer • Drivers call IWiaTransfer::GetNextStream to get streams for the next pages • Drivers should call GetNextStream in a loop, and S_OK indicates another page has been requested • Multiple files can be produced by this method – the application requests the file types, and each page can represent a file; Previously a single transfer resulted in a single file • For multi-page TIFF files, request a single stream and write all the pages to the stream • For multiple file transfers, request a stream for each file

32. Sample Stream-Based Transfer Source Code MyWiaDriver::drvAcquireItemData( BYTE *pWiasContext, LONG lFlags, PMINIDRV_TRANSFER_CONTEXT pmdtc, LONG *plDevErrVal) { // Check what kind of data transfer is requested. if (lFlags & WIA_MINIDRV_TRANSFER_DOWNLOAD) { // This is stream-based download IWiaMiniDrvTransferCallback *pTransferCallback = NULL; hr = pmdtc->pIWiaMiniDrvCallBack->QueryInterface(IID_IWiaMiniDrvTransferCallback, (void**) &pIWiaMiniDrvTransferCallback); if (SUCCEEDED(hr)) { IStream *pDestination = NULL; // Get the destination stream from the client hr = pTransferCallback->GetNextStream(0, bstrItemName, bstrFullItemName, &pDestination);

33. Sample Stream-Based Transfer Source Code if (hr == S_OK) { BYTE *pBuffer = ... ULONG ulBytesRead = 0; // Read the next chunk of data into the buffer while(GetNextDataBand(pBuffer, &ulBytesRead)) { // Write the data to the destination. // This stream could be a file, memory, // or anything else. The driver does not need to know. hr = pDestination->Write(...); if (SUCCEEDED(hr)) { // Send progress hr = pTransferCallback->SendMessage(...) } } // Etc.

34. NT Authority\LocalService Security Context

35. NT Authority\LocalService Security Context • Imaging drivers run under LocalService account (Windows Server 2003 and Longhorn) • Assumptions about access to resources can cause “Access denied” errors under new security context • Typical areas affected • File system access • Registry access • Named kernel objects • Out-of-process COM objects • Creating and opening other processes • Many problems can be solved by putting correct security descriptors on objects • To share objects with other components • Create a discretionary access control list (DACL) that allows access to the object

36. NT Authority\LocalService Security Context • File system access • Under LocalSystem in Windows XP, driver had Read/Write access to almost any directory • Under LocalService, driver has mainly Read access only • If driver only reads files, then usually not a problem • Drivers should create or write their private files in the %userprofile% directory • Note: the user account in this case is “LocalService” • Registry Access • Much of the registry is available for the driver to read • However, WIA drivers should write only to the DeviceData section of the registry key that is handed to them in the IStiUSD::Initialize entry point

37. NT Authority\LocalService Security Context • Named kernel objects • Two main problems • Namespaces • Access permissions • Applications and drivers may run in different sessions • Each has its own namespace • Use the correct namespace when specifying the object name e.g., Global\MyDeviceLock • Set the object permissions correctly when creating the object • Don’t rely on default permissions

38. NT Authority\LocalService Security Context • Out-of-process COM components • Require two types of permissions • Launch permissions • Access permissions • Ensure the permissions on the COM objects are set correctly • Use the COM supplied method of storing your permissions under the AppID key • Call CoInitializeSecurity with the EOAC_APPID • This allows administrators to adjust permissions if necessary

39. NT Authority\LocalService Security Context • Creating and opening processes • WIA drivers should not manually start other processes on the system • In Fast User Switching or terminal Services environment, it is often difficult to say who the “user” is • In Windows XP, driver could launch the process under the incorrect account • Generally not possible under LocalService account, unless… • Activating an installed COM component • Windows NT service, etc. • For more information, see the resourcessection

40. Cleanly Supporting Cancel Operations

41. Cleanly Supporting Cancel Operations 1. Driver issues read command to scanner 2. User cancels request 3. Driver only responds to cancel after I/O completes. User is forced to wait… Time

42. Cleanly Supporting Cancel Operations • Minidriver WIA driver gets notification which can cancel outstanding call • IWiaMiniDrv::drvNotifyPnPEvent called with WIA_EVENT_CANCEL_IO • Take extra care as it can be called any time • It should be treated like an interrupt routine

43. Cleanly Supporting Cancel Operations RawReadData(..) { …… ReadFileEx(hDevice,..,..,&overlapped); dwWait = WaitForMultipleObjects(&HandleArray); if ( dwWait == GlobalCancelEventIndex) { // The cancel event was set, so we are // requested to Cancel I/O; } else if ( dwWait == IOCompletedIndex) { // I/O request completed successfully…… } else … } drvNotifyPnPEvent(..) { if(*pGUID==WIA_EVENT_CANCEL_IO){ // Signal the cancel event so // our RawRead doesn’t wait for // I/O completion SetEvent(hCancelEvent); } else { //................ } }

44. Correctly Writing And Validating WIA Properties

45. Correctly Writing And Validating WIA Properties • How do I know which properties the application is writing? • Driver is told by PROPSPEC parameter in drvValidateItemProperties • E.g., If application writes Horizontal and Vertical resolution, args to driver are:nPropSpec = 2pPropSpec = {WIA_IPA_XRES, WIA_IPA_YRES} • Why is this important? • Driver is required to check the correctness of the properties being written and update dependant properties/valid values • If more than one property is being written • Driver should validate all independent properties first • Then update dependant valid values • Lastly, validate the dependant property values the application wrote

46. Correctly Writing And Validating WIA Properties • Example, driver may support • If current TYMED is TYMED_CALLBACK, valid values for WIA_IPA_FORMAT only has BMP • But if App writes TYMED_FILE and WiaImgFmt_JPEG in one call, this should succeed

47. Correctly Writing And Validating WIA Properties • The way to do this is • Driver validates independent property first i.e., • WIA_IPA_TYMED set to TYMED_FILE is supported • Then update valid values for dependant property i.e., • WIA_IPA_FORMAT valid value should change to {WiaImgFmt_BMP, WiaImgFmt_JPEG} • Lastly, validate dependant property • WiaImgFmt_JPEG is now valid for WIA_IPA_FORMAT, so it should succeed

48. Correctly Writing And Validating WIA Properties • Driver should also update any dependant properties not being directly written by application • Example • If Horizontal Resolution = 100 dpi, Horizontal Extent for 3 inch width is 300 pixels • If Horizontal Resolution changes to 200 dpi, Horizontal Extent must change to 600 to still represent a 3 inch width • Remember: properties being directly written are indicated by PROPSPEC parameter in drvValidateItemProperties

49. Summary

50. Summary • Longhorn drivers support a new item tree structure • Check our NDA site regularly for new white papers • The application compatibility layer will guarantee that Longhorn drivers will work with legacy applications, and that legacy drivers will work with Longhorn applications • The preview component will work for Legacy Applications, the Image Segmentation Filter will not • Any driver which does not work under NT Authority\LocalService (legacy or new) WILL NOT WORK on Longhorn • Stream-based transfers will be supported in Longhorn • Legacy transfers must also be supported by Longhorn drivers