DataShare

1. DataShare New York City’s Enterprise Integration Platform Michael Fiorito, Kate Fitta NYC Mayor’s Office of the Criminal Justice Coordinator Kris Carpenter, Jon Natarajan NYC Department of Information Technology and Telecommunications

2. Agenda DataShare Overview – Progress to Date

3. DataShare Overview – Progress to Date • Michael Fiorito • NYC Mayor’s Office of • the Criminal Justice Coordinator (CJC)

4. Rewind to 2003… • Criminal justice agency CIO’s, CJC, and DoITT convened a workgroup to discuss the possibility of introducing new technology to meet their business needs. • The group produced a report that summarized their research and outlined possibilities for new technology. • The group also recommended hiring a small team of dedicated CJC and DoITT staff to coordinate the overall effort (i.e., a project team) and build a sustainable integration program.

5. DataShare Overview • Why not replace an existing mainframe system utilized by criminal justice agencies (DataShare 1.0) with a scalable enterprise solution? • DataShare was designed for citywide integration, but deployed initially for 17 agencies (city, state, not-for-profits). • The Public Safety Portal, which is the user-facing dimension of the City’s integration solution for criminal justice agencies, includes both administrative and business user functions. • The initial phase concluded in July 2006 with the deployment of the Public Safety Portal and a set of GJXDM exchanges. • This undertaking was a Mayoral initiative, but agencies did not have a mandate to participate

6. Phase 1 – Facts & Figures • 12 months for requirements gathering, solution analysis, systems integration procurement and solution selection • 18 months to complete implementation – 2 major deployments • 17 city, state and nonprofit agencies integrated via hub-and-spoke architecture • 35-40 FTEs comprised the dedicated project team – City staff and consultants combined • 60+ agency-side resources brought to bear during 1st phase • Approximately $12M spent on professional services, HW, SW • 100% of post prod support provided by City of New York upon Phase 1 completion. No ongoing costs.

7. DataShare – Current Landscape Providers Receivers Public Safety Portal Public Safety Portal 18B Web (Assigned Counsel) • Juvenile Case Management System • Law Department • Dept. of Probation • Dept. of Juvenile Justice (DJJ) NYPD Special Narcotics Prosecutor Dept. of Corrections DataShare Queens County DA Criminal Justice Agency Kings County DA Office Court Administrator Bronx County DA Manhattan DA Richmond County DA Legal Aid Society

8. Phase 1 – Scope and Deliverables • Phase 1 Receivers • NYPD • Department of Probation • Department of Correction • District Attorneys (5) • Special Narcotics Prosecutor • CJA • Law Department • Dept. of Juvenile Justice (DJJ) • OCA • Legal Aid Society • Assigned Counsel Plan • HRA • DoITT (on behalf of CJIS) • Phase 1 Providers • NYPD • NYC DOC • Criminal Justice Agency (CJA) • OCA Agencies connected to new DataShare infrastructure and providing or receiving at least one data exchange via the hub or portal Agency Participation • Core Hub Features & Functionality • Push functionality • Pull functionality • Query application • Notification application Criminal Justice Integration Platform (Hub) The behind-the-scenes, criminal justice segment of the Citywide integration environment hosted by DoITT • Phase 1 Business Deliverables • NYPD arrest data push (legacy & GJXDM) • NYPD arrest notification (GJXDM) • CJA ROR interview push and pull (GJXDM) • OCA supreme and criminal court actions push (legacy & GJXDM) • DOC inmate information (legacy & GJXDM) • DOC release notification (GJXDM) • DOC jail status query (GJXDM) • CJA NYSID – SSN match (legacy & GJXDM) Criminal justice data that is shared between and among participating agencies via the hub and portal Data Exchanges/Services • Core Portal Features & Functionality • Functionality for agencies to subscribe to notifications • Functionality for agencies to perform queries • Functionality for agencies to pull data • System administration for agencies • Reduced sign-on (RSO) for DNA HITS application • e911 application link • webCRIMS application link from OCA • e-Justice portal link from DCJS Public Safety Portal Functionality and Links to Applications The user-facing dimension of the City’s integrated justice solution hosted by DoITT on CityShare

9. Guiding Principles & Strategic Decisions • Decisions about DataShare architecture needed to consider the autonomy of agencies • DataShare had to be architected and designed in such a way that it could be scaled for citywide integration in future phases • DataShare was intended to serve as a pass-through system, with minimal data stored centrally • The deployment of DataShare 2.0 and decommissioning of DataShare 1.0 should have minimal impact on agencies and result in no loss of functionality for agencies. • The City decided to adopt GJXDM as its transport standard for criminal justice services

10. GJXDM Strategy & Approach • Global Justice XML Data Model (GJXDM) is is intended to be a data reference model for the exchange of information within the justice and public safety communities. The Global JXDM is sponsored by the US Department of Justice (DOJ) Office of Justice Programs (OJP), with development supported by the Global Justice Information Sharing Initiative's (Global) XML Structure Task Force (GXSTF). • City decided to implement both legacy and GJXDM versions of existing exchanges via new platform to facilitate agency transition and minimize risk • GJXDM SME consultants worked with provider and receiver agencies to develop schemas and leverage national model as appropriate. • GJXDM workgroup has formed among NYC stakeholder agencies to drive strategy going forward, especially collaboration with state and national initiatives; DOP, NYPD, DOC to play key leadership roles.

11. Portal Functions Available to Business Users

12. Portal Administrative Functions

13. Public Safety Portal Services • Public Safety Portal Services • DOC Jail Status Query – This function enables users to enter a NYSID, book and case number or a combination of first name, last name and date-of-birth via the Public Safety Portal and retrieve the incarceration status and history for the individual(s) that match the criteria. Jail status data includes information about the inmate's current incarceration status, incarceration history, warrant history and DOC housing history from 1999 - present. • CJA Interview Pull – This function enables users to enter a docket number or arrest number via the Public Safety Portal and retrieve the details of the corresponding interview the Criminal Justice Agency conducted with the defendant prior to their criminal court arraignment. Interview data includes information about the defendant's employment, residence, education and criminal history and is only available for interviews conducted from 2005 - present. • Subscriptions for NYPD Arrest Notification (Basic) - This function enables business users to enter a NYSID, name information or location information via the Public Safety Portal and subscribe to be notified via email when an arrest event happens relative to the individual or location specified by the criteria. • Subscriptions for NYPD Arrest Notification (Advanced) - This function enables administrators to enter criteria corresponding to up to 150 arrest fields via the Public Safety Portal and subscribe to be notified via email when an arrest event happens relative to the specified criteria. • Subscriptions for DOC Release Notification - This function enables users to enter a NYSID or arrest number or book & case number via the Public Safety Portal and subscribe to be notified via email when the corresponding inmate is released from or admitted to City correction custody. • Links to Applications - DNA HITS, e911, WebCRIMS

14. More to Come… • Orchestrating services • Growing the community of participants • Key to Success: Ongoing partnership between DoITT, CJC and participating agencies

15. DataShare Architecture • Kris Carpenter • NYC Department of • Technology & Telecommunications (DoITT)

16. Information Sharing Capabilities • The DataShare & Public Safety Portal solution facilitates the sharing of information across city agencies providing business process integration for the participating agencies. • DataShare will provide the following services related to enterprise applications integration (EAI) • Business process routing (via publish/subscribe) • Data standardization • Web-based portal that can offer truly distributed business applications • Business process auditing capabilities • Field level filtering for messages • Security • 24/7 Monitoring

17. Existing Systems Installed Base • MVS • VAX/VMS • LINUX • UNIX • Microsoft • FTP • SMTP • CICS • Flat Files • DB2 • ADABASE • DATACOM • VSAM • ADMINS/VAX • RDB • CACHE • MUMPS • SQL SERVER • ACCESS • ORACLE • EXCEL

18. Technical Architecture Requirements • Security – Authentication, Authorization, Encryption, LDAP • Performance – Handle 100’s of messages/second • Scalability – Able to add new data exchanges easily • Availability – 99.9% availability as ‘crime takes no holiday’ • Extensibility – Expose functionality via different methods like Web services • Standards – J2EE, LDAP, GJXDM

19. Meeting The Needs – New Environment • Standards-based (J2EE, LDAP, SQL) • Enterprise Service Bus (WebSphere MQ and Broker) • Global JXDM • Public Safety Portal (Vingette & BEA WebLogic ) • Involve the agencies that know the data • Involve the agencies that know the processes • Use existing City application implementation standards

20. DataShare: Message Oriented Middleware Hub and Spoke Architecture

21. DataShare: The Solution The Solution consists of 6 key components

22. Architecture & Components • Agency Adapter Platform • Agency adapter programs and WebSphere MQ that is utilized as a gateway between the criminal justice agencies and the enterprise service bus hosted at DoITT. • Enterprise Service Bus (Message Broker) • Integration layer that provides message routing and integration business services for citywide agencies. • Novell eDirectory (DoITT’s enterprise directory server) • Enterprise directory server that stores citywide agency users and groups. It’s used to provide services such as distributed provisioning, authentication and authorization. • Database • Storage of application and transactional metadata. • Public Safety Portal • Web front end that provides access to web applications (such as subscriptions and queries), user administration, and reports. • Notification Services • Services provided to the Portal users, and the backend systems of criminal justice agencies, to subscribe to key transactions traversing the Enterprise Service Bus.

23. Adapter Approaches • Adapters key to joining systems: • Used in EAI to be non invasive to systems • Concept used in SOA as a Wrapper • Allow existing formats to be utilized: • Suppliers convert to a standard • Receivers convert standard to legacy format for consumption • Access supplied and/or received messages in an existing fashion: • File System • Queue • Database • Other Storage mechanism

24. Enhancements Underway Currently (4-6 months) • Low hanging fruit • Federated query – giving users the ability to perform searches on NYPD, CJA, and DOC and retrieve a consolidated view of a person or event by entering a single set of criteria. The federated query capability will be greatly enhanced as more services are deployed through DataShare. • Service orchestration – when a user performs a DOC jail status query, they are given the option to initiate a DOC release or admit subscription from the jail status query output screen • Geocoding – exposing existing GIS services to the criminal justice participants via Datashare – incorporating a global address system.

25. Phase 2 Gameplan (12 – 18 months) • Grow the list of agency participants • We have some new agencies coming on as providers: DCJS and DOI • Some current receivers will become providers: DOP, BXDA, DANY, QCDA, LAW, DJJ, SNP, LAS • Some agencies/entities will become new receivers: ACS, HIDTA, OCFS • Integrate services deployed through DataShare into applications: DNA HITS, eArraignment • Deploy 15 – 20 new services/exchanges through Datashare • Working on delivering a host of new data exchanges with existing and new agencies • Goal is to build on what’s been deployed

26. Building the Highway • Highway has been built • Datashare is an enterprise system built to grow with the addition of new services and new providers and incorporate diverse environments • Now it gets really interesting

27. Contacts • Non-Criminal Justice Agencies • Jon Natarajan, DoITT (jnatarajan@doitt.nyc.gov) • Kris Carpenter, DoITT (kcarpenter@doitt.nyc.gov) • Criminal Justice Agencies • Michael Fiorito, Mayor’s Office (CJC) (mfiorito@cityhall.nyc.gov) • Kate Fitta, Mayor’s Office (CJC) (kfitta@cityhall.nyc.gov)