Tribeca streams are like RDBMS relations with one important restriction: Tribeca streams may not be joined. Arbitrary comparisons between records in streams are not allowed for performance reasons. A join operator would permit users to express queries that could be executed only by sorting or repeatedly passing over the data on tape.
The Tribeca window filter, however, is a restricted form of join operation that relates records in a window to records in a stream. At any given moment during the execution of a query, the records in a window can be compared to the current record in any stream. For example, the query below searches for cells whose VCI has appeared in a recent predecessor cell. It defines a moving window that records the last 100 atm cells seen, then compares the VCI of the current cell to those of the cells in the window.
stream_proj {s1.atm.vci} p1
stream_window w1 on p1 defined by {s1.count 100} is moving
window_filter {{s1.atm.vci.eq w1}}
{s1.vci s1.ts w1.ts} r1
The window filter arguments are a list of join predicates and a list
of projected fields.
While users cannot compare arbitrary records in streams, window filters allow users to compare records that are temporally near one another. Window filters are not as powerful as true joins, but are useful in traffic analysis and can be executed efficiently as long as the window is small enough to fit in memory. For example, one traffic analysis query takes the mean and standard deviation of packet length over N seconds, then uses a window filter to search for packets that are significantly larger than the mean over the next N seconds. We also allow users to initialize windows from files so they can do things like select UDP packets destined for port numbers listed in a file.