Add an option to receive real-time data in client during buffer recovery
Add an option to receive real-time data in the client during recovery of buffered data:
The default behaviour of our interfaces / buffering is that after a connection failure the buffer will be flushed and data will be sent in it’s original order (which is correct behaviour). During that time, there is no view on the realtime (snapshot) data for end users.
Customer is asking for an option to not have to wait for the ‘new’ snapshot data.
It is clear that this would produce many OOO events, which will influence PI Data Archive performance and most probably will influence any analytic/report/visualization on top of those tags.
Perhaps there can be a way to have a parallel dataflow to the visualization layer, showing 'new' data whilst buffered data flows through the Data Archive processes in the background?
In response to Rene Thomassen, "That is known and clear to the customer...."
Yes Rene, exactly! in order to see the data, as soon as the communication channel is available...
the function could be configured/enabled through a specific parameter with several options: e.g. send only selected TAGs (in a specific location) with a different sampling rate (e.g. advise, every 10",30" or 10'....) also to reduce the impact on the bandwigth...
the data can be discarded since that the "legacy PI data flow" will be still on going
Rene Thomassen commented
In response to Gabriel Michaud-Verreault, "This would significantly negatively impa..."
That is known and clear to the customer. However this is still a valid functional ask, and perhaps there can be some creativity - e.g. in the form of a separate visualization of the datastream coming from the interface node (separately - not entering the compression and archiving process). Of course, this is just an idea.
Gabriel Verreault commented
This would significantly negatively impact the system as a whole.
When data arrive at the PI Data Archive in order, the PI Archive subsystem adds the value to the end of a point's record in the archive file. However, when data come to the PI Data Archive out of order, the PI Archive subsystem must continuously move aside existing data to make room for the old data. This puts an extra load on the PI Archive subsystem and can make it less responsive to other tasks.
Furthermore, out of order events are not filtered by compression. Therefore, out of order data result in more events being stored in the archives than if the events were in order and filtered by compression.
Out of order data can also impact calculated tags such as Performance Equation (PE) tags and Totalizers. If source tags for your calculated tags receive events that are timestamped earlier than the snapshot timestamp, these calculated tags may not include those events in their current calculation. Since Totalizers only incorporate snapshot data in their calculations, all out of order data will be not be incorporated into their calculations
It would be very useful to have this functionality!