Dealing with Big Data

Documents, mail, correspondence, RSS feeds, and archival data are just one component of big data. Metadata (the "data around the data") is becoming increasingly important as another attribute or filter. Context, sentiment, and even geospatial relationships all provide different, additional elements of value. The challenge is mining these elements without being overwhelmed by extraneous and repetitive big data.

CAAT provides its Intelligence partners with a range of capabilities to categorize big data and to gain rapid insights from it. Analysts "train" CAAT by providing sample sets of documents, called exemplars, that contain either concepts or phrases that the analysts want to correlate or find. Armed with this basic information, CAAT is then ready to categorize big data and pick out only those items that are relevant to the identified concepts and categories.

A single instance of CAAT can categorize millions of items an hour, easily exceeding the largest big-data challenges.With scores of 90% on the Reuters test data, CAAT's precision and recall accuracy is second to none.