• FINDER ETL tools are called FINDER Parsers or simply parsers.
• They are custom written for every RMS, JMS, CAD, Pawn and other system used by law enforcement agencies (hereinafter referred to as “LEA System” or “source system”.)
• A FINDER Parser is a lightweight Windows console application.
• A parser can be installed on any Windows machine continuously powered on, including a laptop.
• It is installed at a location on the agency’s network from where it can open a connection to both the RMS database and the FINDER database. Typically, this is behind the agency’s firewall.
• Two commercial off-the-shelf (“COTS”) components required for every parser are: the LEA System database driver and the free Microsoft SQL server database driver. Most database drivers are available for free or at a minimal cost.
• The parser is scheduled to run at regular intervals using the Windows task scheduler.
• The parser connects to the LEA system database using the respective database driver, extracts new and updated reports using the last updated date, transforms the data and inserts it into the FINDER Elasticsearch database using the Elasticsearch server driver (see illustration above).
• Data transformation includes data cleansing, normalization, filtering and expunging described in detail in subsequent sections.

Data flagged as confidential in the source is not read by FINDER.

If data is marked as confidential after it has been inserted in the destination, it will then be deleted from the destination.

Data normalization is the process of injecting structure and order into the data. FINDER parsers extract data entities (persons, vehicles, cases, etc.) from structured data sources like databases and from unstructured data sources like text files. In either case, the data is transformed to the FINDER structure and loaded into FINDER.

Besides injecting structure in data entities, data elements like race, sex, location, etc., are structured using the following methods:

• Code mapping: Codes like hair color, eye color, race, sex, etc. are mapped to a standard code like FCIC or NCIC.
• Standardization: Data fields can be standardized for consistency. For example, “WEST” can be abbreviated to “W”; full names can be formatted as last name, first name, middle initial, etc.
• Geocoding.

Data cleansing is an attempt to fix or filter some obvious errors and abnormalities. 

FINDER parsers perform the following cleansing operations during ETL:

• Mandatory Constraints: Certain columns cannot be null.
• Data Types: Values in particular columns are required to be of a certain data type like Boolean, numeric, etc.
• Ranges: Certain values are required to fall within certain ranges. For example, dates can fall between 1/1/1700 to 12/31/9999.
• Regular expressions: Certain values might have to be parsed in a certain format. For example, SSN might have to be parsed with dashes, etc.

Data cleansing should not be confused with data accuracy and completeness. Spelling mistakes and missing information cannot be fixed during ETL as it requires going back to the report writer. When agencies correct the reports in their systems, the FINDER parser will sync the changes to FINDER.

FINDER’s monitoring system receives status updates and error notifications from parsers installed at agencies. When error notifications are received, FINDER will contact the agency to resolve those errors. Additionally, parsers can also be configured to send notifications to the agency’s IT contact.