Starting to Audit Documents in the Engineering Domain

1. Starting to Audit Documents in the Engineering Domain Peter J Wild, Steve Culley, Chris McMahon, Mansur Darlington, Shaofeng Liu Innovative Manufacturing Research Centre Department of Mechanical Engineering University of Bath

2. Presentation Overview • Project Context • Introduction • Study: Setting and Context • Interim Findings • Modelling Framework • Conclusions and Future Work

3. Project Context • ICID Information Content in Documents • Three strand project: • Stream A: Document Structures & Decomposition Schemes • Stream C: Computational Methods & Mark-up Languages • Stream B: Engineers’ Exploration Behaviour & Document Use Patterns • Document Audits • Diary Studies • Observational work • Leading to evaluation / experiments using • Document structures (Stream A) • Scenarios of use (Stream B) • Tools (Stream C)

4. Document Studies and Auditing • Interest in • how people use, generate, save, and search, paper and electronic based documents • auditing information flows • Attempting to pull this into a coherent approach to auditing engineering documents • Starting questions for the auditing process • What do ‘they’ haveuse and generate

5. Audits should provide data about • Varieties and types of engineering documents; • Strengths and weakness of document manifestations • Patterns of document use within different types of engineering company (e.g. SME, design, manufacturing)

6. Case Study: Setting and Context • TrollCo • Engineering Design and Manufacturing company based in Wiltshire • General engineering work • e.g. nuclear, railway, highways • Design and manufacture of their own product line • Culture of pride in the quality of their work • “We are the Rolls Royce of…..” • Pressures of the market • “Trouble is people want Rolls Royce at Ford prices”… • British Standards Institute (BSI) quality certified

7. Research Approach • Engineers would volunteer stories, scenarios, information about how things were carried out • Lacking ‘formal’ interviews • Informal interviews, and scenario gathering • Pen and paper observations • Iterative discussions with TrollCo’s engineers have taken place • Audits of physical & computer based documents

8. What Do They Have? • 1000s Documents • 250+ ‘Types’ • Reflect a variety of concerns, manifestations and life-spans • Expected and TrollCo specific • Cannot be easily wedged into a simple classification (e.g. ISO 9001) • We are exploring classification methods

9. Using and Generating • Use and generate large number of documents. • Mostly developed by the quality management system • Many generic to product and engineering services division • Distinction between routine and exception documents

10. Product Division Documents (Order)

11. TrollCo Consider Their Document Usage Chaotic! • Documents have: • needed information • layout and meaning are comprehensible • Document ‘chaos’ appears to stem from: • ambiguity in orders • dealing with multiple instances of orders • the variety inherent in the product line and customer needs • Design Documents probably are chaotic

12. An Unused Catalogue Taxonomy • Files marked with a category and subcategory • Taxonomy was locally developed and reflected the forms of material that the engineers used, and still use today • It was apparent that it was neglected

13. Contemporary storage situation

14. And

15. Why? • Demise of the ordered catalogue store and taxonomy due to: • Time • Location • Supplier management strategy

16. Paper vs. Electronic Documents / Files • Minimal use of bookmarks • Email client file structure was default • Naming and hierarchy conventions in files are variable and inconsistent between electronic and paper manifestations • Struck by paucity of manifestation, binding and container mechanisms in computer based file systems

17. Paper vs. Electronic Documents / Files • In future audits we will investigate further the semantics ascribed to document ‘bindings’ • More generally we could investigate more elaborate binding mechanisms in computer based documents (see also Rogers 1996)

18. A Modelling Framework? • Activity being undertaken with documents is essential to understanding why they exist and how they are used • Non-partisan approach to how we model the work aspects in analysis of data from document audits • Analysing the context of documents at the five levels of Culture, Flow, Sequence, Artifact, and Physical Character have been influential in framing our understanding

19. Culture Flow and Sequence

20. Artefact and Physical Properties

21. Conclusions • Work in progress • Interim findings • Have Use and Generate • Catalogue Taxonomy and Storage • Electronic and Physical Manifestations • Focussing on 3 questions of have, use, and generate, and five levels of analysis Culture, Flow, Sequence, Artifact, and Physical Location • Are Have Use and Generate always that distinguishable? • Do engineers think well in these terms?

22. Future Work • Arranging to audit documents in other companies. • Particular focus in future audits is adding methodological rigour to the data gathering • Also using ‘auditing’ software tools to unobtrusively gather data and speed up some analysis, and provide more robust statistical comparisons between organisations.

23. Questions and Discussion