/*
 * Gargoyle Client Software. Tools to read, analyze and manage Argus data.
 * Copyright (c) 2000-2024 QoSient, LLC
 * All rights reserved.
 *
 * THE ACCOMPANYING PROGRAM IS PROPRIETARY SOFTWARE OF QoSIENT, LLC,
 * AND CANNOT BE USED, DISTRIBUTED, COPIED OR MODIFIED WITHOUT
 * EXPRESS PERMISSION OF QoSIENT, LLC.
 *
 * QOSIENT, LLC DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS
 * SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
 * AND FITNESS, IN NO EVENT SHALL QOSIENT, LLC BE LIABLE FOR ANY
 * SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
 * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER
 * IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
 * ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
 * THIS SOFTWARE.
 *
 */

/*
 *  ratod - tools used to develop time of day behavioral flow data
 *          The essence of the effort is to generate the canonical day,
 *          through statistical process of 1,000s of days, and comparing
 *          data presentations.
 *          
 *          The first concept is to remove the time from the flows, so
 *          for daily processing, we'll set the starting time to be
 *          0 by subtracting the starting second for the specific day,
 *          and all other time is offset by this value.
 *          
 *          Then, we'll bin the data to generate 5m inventories.
 *          While this maybe a lot of data, we'll do some experiments to
 *          see what works ...
 *          
 *          
 *  
 *  Author: Carter Bullard carter@qosient.com
 */

NOTES

OK, so currently the implementation provides rabins() behavior
with time normalization to Jan 1, 1970.  This provides the
greatest flexibility to generate aggregated or unaggregated
data.  Right now we're doing Time of Whatever normalization.
Using the time bins command line configuration data to
specify arbitrary time period normalization.

The easiest support for ToD processing is to stream data
through a tool that only does time normalization, but
putting the rabins() aggregation does provide most of
what we need in a tool like this.

Simple normalization of the time is not enough to provide usable
data, as direction information is critical to making useful TOD
flow run data.

The concept to date is to provide time normalized 300sec resolved
L2/L3 service oriented flow run data.  This will be read in using
the " -M rtime " options, so that it represents a TOD cache of
expected network behavior.  The flow records can span the entire
day, if, for example, there are no idle times, or it can be broken
down into 300 sec bins.  

NOTE - Racurses() will need to manage its timeout values based on
       " ltime " for this to work.


Service oriented flow run data
Normalizing a set of flows into a daily canonical flow run dataset,
involes removing the ephemeral data, such as source port values.
This demands that service direction be a part of the process, so
as to minimize ephemeral collision (where ephemeral data is inadvertently
introduced into the comparison sets).

The goal is identify new flows that are outside the services box,
as well as identify when expected flows do not show up.
The primary motivator in the ratod() analytics.

